Keeping AI Fraud Decisions Auditable: What a Model-Risk Reviewer Needs

The question that defines the requirement

A model-risk reviewer or an auditor picks one decision from six weeks ago and asks: reproduce it. Same transaction, same feature values as of that moment, same model version, same score. If you can't, no explanation technique rescues you. A fraud model audit trail for compliance is, first, an engineering requirement about reconstructability — and it shapes the architecture as much as the model does.

What has to be logged, per decision

• Feature values, not names. The record stores the actual numbers fed to the model — device_age_hours: 2, amount_vs_90d_median: 4.1 — not a list of feature names and a score.
• The model version. Exactly which trained snapshot produced this score.
• The prompt version and model snapshot, if a reasoning layer contributed.
• The policy and threshold version in effect at that moment.
• The score, the routing outcome, and any analyst override with its stated rationale.

Where it's stored

Off the request's hot path, written asynchronously to an append-only, immutable store, retained for the full regulatory window. On our fintech fraud-scoring project reconstructability was the hard requirement — a decision a reviewer couldn't rebuild later was treated as a defect — and it drove the event-log design more than any latency number did.

Append-only matters as much as retained. If a record can be edited after the fact, its evidentiary value drops to zero — a reviewer has to trust that what they're reading is what the system actually did, not a tidied-up version. Write once, never update, and keep the write path separate from anything an operator can reach.

The feature-store trap

If your online feature store has updated since the decision — and it updates continuously — you cannot reconstruct the inputs by querying it now. The only fix is to snapshot the actual feature values into the decision record at scoring time. A pointer to a mutable store is not an audit trail.

The LLM-layer trap

A fluent rationale is not a faithful one. An LLM will produce a plausible story that doesn't match what moved the score. If a reasoning layer is in the decision, its narrative has to be anchored to the same structured feature contributions the score came from, and logged with the prompt version, so a reviewer can check the story against the math.

The ensemble trap

A learned combination of two model outputs is harder to explain than either output alone. If regulatory expectations tighten on what has to be interpretable about an automated decision, the combination step may need to be simplified into something more directly readable than a fitted blend of scores — a change to plan for, not to be surprised by.

What a review actually walks through

1. Pull the decision record.
2. Re-run the same inputs against the same model and policy versions; confirm the score matches.
3. Read the reason codes; confirm they reflect the actual top drivers, not a template.
4. Check the override log if a human touched it.

If every step is answerable from the record, the system is auditable. If any step needs a "we think it was roughly…", it isn't.

Where this stops being right

• Scale the rigour to the consequence. A full reconstruction package around a low-value payment decline is wasted effort; a credit denial needs all of it.
• Jurisdiction changes what "explainable" means. This is a prompt for the conversation with counsel, not a substitute for it.
• If you're not regulated and disputes are rare, structured logging of inputs and score may be enough — don't build the audit cathedral you don't need.

FAQ

Is logging the score and the feature names enough? No. You need the feature values as of the decision, or you can't reproduce the score.

Can the LLM write the audit rationale for us? Only if it's grounded in the actual feature contributions and logged with its prompt version. A free-text rationale not tied back to the score is a liability.

How long do we retain decision records? At least your regulatory retention window, in an append-only store — long enough that a dispute or an exam years later still has the record.

ISTRALLEN builds fraud decisioning for regulated fintech teams with per-decision reconstructability designed in from the start; see AI for Fintech.