Context
A system that answers from a rule corpus can cite the rules it used, and citation is commonly treated as sufficient evidence that an answer is grounded. It is not. A citation proves a rule was retrieved and named. It proves nothing about whether the code beneath it obeys that rule.
Nexa is a local, offline assistant that answers Odoo, TypeScript and JavaScript validation questions from a schema-validated rule corpus, running entirely on the user's own hardware. The first version is finished, and this record captures the verification decision that gives it its shape.
Problem
During development the working model cited odoo.onchange-is-not-validation — a real rule identifier, present in the corpus, traceable to upstream documentation — and then wrote an @api.onchange to guard the value. Every citation resolved. The answer was still wrong.
That failure is silent in a way an invented identifier is not. A hallucinated API fails at import, and the tooling already in place catches it. A real rule cited above code that violates it produces something that parses, installs, and simply never enforces the invariant it appears to enforce. The failure surfaces far from the answer that caused it, as missing validation in production rather than as an error at the point of generation.
An answer that names a rule and breaks it is also more dangerous than an answer with no citation at all, because the citation is precisely the signal a reader uses to decide the answer has been checked.
We need a clear boundary between producing an answer and presenting that answer as grounded.
Decision
Generated output will be checked against the rules it cites, and against the rules pinned for its stack, before it is presented.
Two independent checks run on every answer. The first asks whether every cited rule identifier exists in the corpus. The second asks whether the generated code actually obeys the rules that were cited. The second check exists because the first is not enough.
Compliance is machine-checkable wherever a rule can carry a violation pattern, optionally cleared by a satisfied pattern. A failed check escalates the question once to a larger model; a second failure is reported as unverified rather than presented as an answer.
The presentation layer should be able to assume that anything shown as verified has passed the checks that exist — and that the verdict is stated in terms of those specific checks, never as a general claim of correctness.
This establishes verification as a boundary between generation and presentation, rather than treating verification as an incidental responsibility of whoever reads the output.
Alternatives Considered
Trust the Citations
Rejected.
Citation existence proves a rule was named, not that it was obeyed. This was observed directly rather than anticipated: a real rule cited above code violating it, with every citation resolving cleanly.
Rely on Downstream Tooling
Rejected.
Type checkers, linters and test suites catch what their own models can express. The rules in this corpus encode framework semantics that no general-purpose tool knows: a constraint decorator that silently ignores dotted field names while a near-identical decorator accepts them, or a handler that only fires inside a form view and therefore cannot hold a business invariant.
In each case the code is syntactically valid, passes type checking, and installs successfully. The defect only appears as absent behaviour, at which point the low-level failure lacks the context needed to associate it with the generated code that caused it.
Strengthen the Prompt Instead
Rejected.
Prompt tuning plateaued after two iterations. Hardening one Odoo rule fixed the scalar cases and simultaneously broke the Many2one handling that had previously been correct. A small model holds a bounded number of constraints, and past that point additional instruction text trades one error for another rather than removing either.
Instructions also bleed across domains: a single prompt covering both Odoo and TypeScript answered a plainly Odoo question entirely in TypeScript. Prose has no schema, cannot be diffed meaningfully, and cannot be tested for whether it still means what it meant before an edit.
Require a Pattern for Every Rule
Rejected.
Five rules describe conditions a pattern cannot express: two require set comparison between a decorator's arguments and the attributes its body reads, one lives in a diff between module versions, and two depend on context a code snippet does not contain.
Forcing a pattern onto these would produce checkers that fire on correct code. A checker that cries wolf is worse than no checker, because it trains the reader to dismiss it — and it will then be dismissed on the occasion it was right. These rules return no verdict rather than a false pass, and the reason for each is recorded alongside it.
Consequences
Positive
An answer that cites a rule and violates it is caught before it is presented.
The verdict is expressed in terms of specific checks rather than as a general claim of correctness.
The gap in coverage is enumerable rather than unknown: five of thirty-two rules return no verdict, each with a recorded reason.
Patterns become testable artifacts in their own right, using each rule's wrong and correct samples as test data that already existed.
Escalation to a more expensive model has a defined trigger — a failed check — rather than a heuristic or a retry count.
The corpus, the system prompts and the compliance checker cannot drift apart, because the prompts and the retrieval index are generated from the corpus.
A rule that turns out to be wrong is correctable in one place, and the correction reaches generation and checking together.
Trade-offs
A violation pattern is a regular expression over generated code. It catches textual signatures, not semantics, so code that is wrong in a way no pattern describes passes silently.
Compliance is only checked against rules retrieved for the question, plus pinned ones. A rule that was never retrieved is a rule that was never enforced.
A verdict of verified means the checks that exist found nothing. It does not mean correct, and that distinction has to be restated everywhere the verdict appears.
Patterns must be maintained alongside the rules they protect, and tested both against their own samples and against every other rule's correct sample to measure false positives.
Escalation costs throughput, and whether it repays that cost is an open measurement rather than a settled fact.
The corpus itself is a source of error: a rule that is subtly wrong is wrong in the prompt, the index and the checker simultaneously, and the pipeline will report verified throughout.
Evidence & Related Work
This decision is implemented in Nexa, and the work below provides concrete evidence of applying the principle in practice.
Implementation Evidence
Nexa
An offline validation assistant for Odoo and TypeScript/JavaScript. Thirty-two rules, thirteen of them critical, twenty-seven carrying an automated check and twenty-seven traceable to upstream documentation. No model weights ship with it; it orchestrates open models pulled from Ollama's registry.
Architecture and Decision Record
The full record of how the verification pipeline works, every significant decision with the evidence behind it including the ones that were wrong first, and an explicit account of what the checking cannot do.
Pattern Validation and Cross-Checking
The build fails on three pattern faults: one that does not match its own wrong sample, one that fires on its own correct sample, and a clearing pattern that clears its own wrong sample. Cross-checking then fires every pattern at every other rule's correct sample to measure the false-positive rate. The first run found three real defects, including a pattern that correctly flagged a sample in the corpus that did not follow its own rule. There are now zero cross-hits, and a negative test confirms the validator genuinely rejects a deliberately broken pattern.
Distribution
Published as a source-available corpus and checking pipeline rather than as a model.
Publication Evidence
Related Article
A technical article documenting the failure that motivated this decision and the engineering considerations behind it.
LLM Workflow Router
The escalation bound in this pipeline is delegated to declared workflow topology rather than to control flow, as recorded in ADR-002. The router's own validator rejected the first topology written for Nexa — a cycle containing a no-re-entry step is dead by construction — at load time, before any model ran.
Relationship to the Broader Engineering Approach
This ADR is part of a broader engineering principle:
Invalid states should be made difficult or impossible to reach the execution layer.
ADR-001 names AI-agent actions as one of the places that principle should apply. This record is that application.
Where ADR-001 places the boundary between configuration and execution, and ADR-002 places it between a declared workflow topology and the models that topology drives, this decision places it between generation and presentation. The move is the same in each case: an input that cannot be assumed correct is checked at a named boundary, and everything past that boundary is permitted stronger assumptions.
The distinctive difficulty here is that the input being validated is produced by the system itself. Configuration arrives from outside and can be rejected outright. Generated output arrives from a component of the system, arrives fluent, and arrives carrying its own claim to have been checked. A citation is a claim, not a proof, and fluency is not evidence.
The same approach applies wherever a system produces output that asserts its own grounding: retrieval-augmented answering, automated code modification, agent tool selection, and any pipeline where a component's confidence is otherwise taken as a substitute for verification.
Decision Outcome
The verification boundary is considered an intentional architectural boundary.
Future work on this system should preserve this principle where practical:
Question → Retrieval → Generation → Verification → Verified Answer, or an explicit Unverified
rather than:
Question → Generation → Answer
This ADR should be revisited if a checking mechanism stronger than pattern matching becomes practical — a syntax-tree checker is the obvious candidate for the two critical rules that patterns cannot express — or if measurement shows that escalation does not repay its cost, in which case those cases belong in the corpus rather than in a second model pass.