Graceful Degradation: What a Fraud Model Does When the LLM Times Out

Plan for the parts that will fail

A fraud model that only works when every dependency is healthy is a fragile addition to an authorization flow. Graceful degradation in a fraud model means every failure mode has a named fallback, decided in advance, flagged in the record, and counted — so a component going down degrades the decision instead of breaking it.

The failures to plan for

• The LLM API times out, rate-limits, or has an incident.
• The online feature store is slow or unreachable.
• The model-serving layer itself is down.
• A downstream policy service is unavailable.

The named fallback for each

• LLM down → return the gradient-boosted score without it, flagged, so you can measure how often the reasoning layer is missing. On our fraud-scoring project, the boosted baseline and the LLM run concurrently and the LLM is never on the 100%-critical path — losing it drops a signal, not the decision.
• Feature store slow → accept features up to N seconds old, marked stale, rather than blocking the score on a fresh read. A slightly stale feature usually beats a timeout in the authorization path.
• Model service down → fall back to rules plus a tighter threshold.
• Everything degraded → a conservative default policy.

Fail open or fail closed — decide per failure mode

A fraud system that fails open leaks money; one that fails closed blocks good customers during an incident. Neither is universally right. For each failure mode, decide based on amount, customer, and risk appetite: a low-value transaction you can't score might pass; a high-value one might hold for review. Write the decision down so it isn't improvised at 3am.

Make degraded decisions visible

Every fallback path flags the decision and increments a counter. A quiet 5% LLM-timeout rate is a real quality issue, and it should surface as a metric, not hide inside "the system is up." Put a rate and an alert threshold on each path — LLM-missing rate, stale-feature rate, rules-only rate — and review them weekly, because a slow creep is exactly the kind of thing that doesn't page anyone until it's a big number.

The degradation runbook

Write it as a table, before the incident, not during one:

A concrete example of "mixed, by amount": a $30 transaction that can't be fully scored passes with a flag for later review; a $3,000 one holds for a human. The line is a business decision, made once, in daylight.

Test it

Kill each dependency in staging and confirm the fallback path still produces a decision inside the latency budget. Degradation logic that's never been exercised is a guess.

Where this stops being right

• Review-and-clawback flows. If scoring can happen asynchronously and be reversed, a delayed score is acceptable and degradation matters less.
• Simple pipelines with one model and no external calls have little to degrade.
• Over-engineering fallbacks for components that don't fail — measure actual failure rates first, then build for the ones that matter.

FAQ

Should a fraud system fail open or fail closed? Neither universally. Choose per failure mode based on amount and risk appetite, and document the decision.

What happens to the LLM signal when it's down? You return the boosted-tree score without it, flagged, and monitor the rate — a rising LLM-timeout rate is its own alert.

How do we test degradation? Kill each dependency in staging and confirm the fallback still meets the latency budget and produces a sensible decision. If it's never been tested, it doesn't work yet — degradation logic is exactly the code that rots unnoticed because it only runs during incidents.

ISTRALLEN builds fraud scoring with a named, tested fallback for every dependency so an incident degrades the decision instead of breaking checkout; see AI for Fintech.