Watching Conformly audit CivicFlow
CivicFlow is a govtech platform putting AI into production, sure the EU AI Act doesn't apply to them. Scroll down to see what a two-minute audit actually finds — and what each finding means, without the legal jargon.
CivicFlow
Public services, streamlined. · Government services software
“We build tools for a city council. Public sector, small budget — the AI Act can't be aimed at us.”
- Benefits EngineDecides who qualifies for support paymentsdeclared
- Citizen botAnswers residents' questionsdeclared
- Risk FlagsScores residents' likelihood of offendingundeclared
- Form SorterRoutes incoming forms to departmentsundeclared
- Fraud ScreenFlags suspicious benefit claimsundeclared
The founder named 2. A read-only scan of the code found 5. Here is what each one means — in plain words.
Resident Risk Flags
Scores individuals for how likely they are to commit an offence, from profiling.
Banned. Predicting whether a specific person will commit a crime, based on profiling them, is a prohibited practice. It has to stop.
In plain words: You can't have AI stamp a 'likely to break the law' label on people. The law forbids it.
Art. 5Article 5(1)(d) prohibits AI risk assessments predicting the likelihood of a person committing a criminal offence based solely on profiling or personality traits. In force since 2 February 2025.
Switch it off. Assessing an individual's crime risk by profiling has no compliant form here.
Conformly flags it as prohibited with the article and reason.
An inspector would open with: "Show every score you assign to a resident and what decision it influences."
Benefits Engine
Decides eligibility for public support payments.
Deciding who gets public benefits shapes people's lives, so it's high-risk. Keep it — but prove it's fair, keep officials able to override it, document it, and (as a public body) register it in the EU database.
In plain words: If software decides someone's benefits, it has to be fair, checkable, and on the public record.
Annex IIIAnnex III(5)(a) lists AI used by public authorities to evaluate eligibility for essential public assistance benefits and services as high-risk; public deployers must also register it under Article 49.
Run the high-risk checklist, add a human-override path, and prepare the EU database registration.
Conformly builds the obligation list, the Annex IV file, and the Article 49 registration pack.
An inspector would open with: "Provide the fairness testing for the eligibility model and its EU-database registration."
Citizen Bot
Answers residents' questions about council services.
Tell residents it's an AI. One clear line.
In plain words: A council helpline bot is fine — as long as it says it's a bot.
Art. 50Article 50(1) requires people to be told they're interacting with AI. Applies from 2 August 2026.
Add "You're chatting with the CivicFlow AI assistant" to the chat.
Conformly hosts the transparency page and the disclosure chip.
Fraud Screen
Flags suspicious benefit claims for a human to review.
Detecting suspicious claims is fraud detection, which the Act does not treat as high-risk. Keep a human making the final call and you're fine.
In plain words: Spotting likely fraud for a person to check isn't the same as deciding someone's benefits. It's fine.
—Fraud detection is outside the Annex III high-risk categories — minimal-risk, provided it only flags for human review.
No action required. Keep the human-in-the-loop and keep it listed.
Conformly records it and the human-review control, so the carve-out is documented.
Form Sorter
Routes incoming forms to the right department.
Nothing owed. It's an admin router; it doesn't decide anything about a person. Minimal risk.
In plain words: Sending a form to the right desk doesn't affect anyone's rights. It's fine.
—Not high-risk and outside Article 50 — minimal-risk only.
No action required.
Kept in your inventory so the record is complete.
The verdict
CivicFlow assumed a small public-sector team was beneath the Act's notice. The audit found a resident-scoring feature that had to stop, a benefits engine needing a full file plus EU-database registration, and one disclosure fix.
That startup was made up. Your AI isn't.
The same deterministic engine can classify your real systems in two minutes — no sign-up, no sales call. Or drop the read-only scanner into your codebase and see what it finds.
npx conformly-scan --dry-run
An example audit — a realistic walkthrough of how Conformly's deterministic engine classifies this kind of AI product. Not legal advice.