How to Write an AI Requirements Doc Engineers Can Build From

"Add AI to X" is not a requirement

A requirements doc that says "the agent should be helpful and accurate" gives an engineer nothing to build against and nothing to test. An AI requirements document has to pin down the task, the inputs and outputs, what "done" means as a measurable bar, and how the system behaves when things go wrong.

The task, with examples

State the task in one sentence — "classify each inbound email into one of these eight intents and extract the order number if present." Then give ten real examples: the input, the expected output, and why. Examples resolve the ambiguity that prose can't, and they become the seed of the evaluation set.

Inputs and outputs, precisely

• Inputs: exactly what the system receives — the fields, the format, where they come from, what can be null.
• Outputs: the schema, the allowed values, the structure. If it's a tool call, the tool's exact signature.
• The boundary: what the system does not return or decide.

Acceptance criteria as an eval-set spec

"Done" is a number on a fixed test set, not a vibe. Specify: the metric (resolution rate, precision and recall, schema-valid rate, relevance), the threshold, the size and composition of the eval set, and where the labelled examples come from. This section is the contract — the build is finished when it clears that bar.

Non-functional constraints

The requirements that actually shape the architecture:

• Latency budget — request-path with a number, or async?
• Reconstructability — does a decision need rebuilding months later?
• Concurrency — peak simultaneous sessions?
• Data residency, retention, PII handling.

Failure behaviour

Not an afterthought — a section. What happens when the model API times out, a tool errors, retrieval comes back empty, confidence is low. Fail open, fail closed, degrade to rules, escalate to a human — specify it per failure mode.

Out of scope

An explicit list. Personalization, a second language, multi-region, the exotic edge case — if it's not in this release, name it, so adding it later is a decision and not a mid-build surprise.

Data sources and readiness

For each data source the feature needs: is there a usable API, is it fresh at read time, are there labelled outcomes, is there enough history. The pipeline write-ups in our portfolio open with exactly this — the constraints and data assessment before any model is chosen, because they decide the timeline.

The doc is a contract, not a wishlist

The acceptance-criteria section is the part both sides sign. "Helpful and accurate" isn't signable — there's no state of the world where it's demonstrably met. "At least 85% intent accuracy on this 200-case set, schema-valid on every tool call, p95 latency under 800ms" is signable, testable, and it's the thing that says the build is finished. If you can't write that sentence, the requirements aren't done yet.

Keep it living

The doc and the evaluation set evolve together. Every production surprise — a failure mode nobody anticipated — becomes a new example in the doc and a new case in the eval set, so the same thing can't ship again silently. A requirements doc that's frozen at kickoff is a snapshot of what you knew before you learned anything.

Where this stops being right

• A research spike — the "requirement" is a question to answer; a full doc is premature.
• A tiny, well-bounded feature with no integrations or compliance angle can be a paragraph plus five examples.
• A genuinely exploratory product where the task itself is still being discovered — write the doc after the first prototype, not before.

FAQ

What's the one section engineers ask for most? The acceptance criteria as an eval-set spec — the exact metric, threshold, and test set that define "done."

How many examples should the doc include? Ten or more real ones, with expected outputs and the reasoning. They resolve ambiguity and become the start of the evaluation set.

Why a separate failure-behaviour section? Because "what happens when it breaks" drives real architecture, and if it's not specified it gets improvised — usually badly, under pressure.

ISTRALLEN writes AI requirements docs with examples, an eval-set spec, and a failure-behaviour section engineers can build from — see what we do.