Fintech

Rules Engine vs LLM Fraud Scoring: What Actually Changes

August 2026 · ISTRALLEN Team

You're not starting from zero

Most fraud teams don't choose between a rules engine and a model — they already run the rules engine, and it's been accreting if statements for years. Block if amount > $500 and shipping country not equal to billing country and card age < 30 days. It works until fraudsters learn the thresholds, good customers keep tripping them, and every new rule quietly breaks two old ones. The case for adding gradient-boosted scoring, and lately an LLM layer, is real — but the rules engine vs LLM fraud scoring comparison is narrower than the pitch, and worth doing one layer at a time.

A transaction is scored by the rules engine first, and a clear match there — sanctions, a known-bad BIN, a velocity breach — is an instant accept or reject on the spot, because rules are deterministic, instantly explainable, and hotfixable in minutes; no model should override them. Only the grey zone continues to a boosted model, which is where the real shift happens: thresholds become gradients learned from labels, and new patterns get handled by retraining instead of hand-editing logic, producing a calibrated score for the cases a fixed cutoff can't capture. Most of that traffic goes straight to a policy layer that tunes thresholds to the business's own loss trade-off and logs the decision. The first dashed line is the selective exception: only borderline scores route through an LLM layer, which doesn't replace the boosted classifier — it adds narrative context and a readable rationale from text the tabular model can't ingest — before the case still lands in the same policy layer. The highest-risk band gets one more dashed step: it routes to human review rather than an automated call, which is the one decision this architecture never makes on its own. And the loop doesn't end at the decision — a third dashed line carries delayed chargeback labels back into the boosted model, because confirmed fraud arrives weeks late and retraining runs on that lag as an ongoing operation, not a launch task. One honest simplification: that dashed line is shorthand for a correlation, not a literal signal path — the chargeback itself arrives later from the card network or issuer and gets matched back to the original transaction's features, rather than being emitted by the decision box directly.

What the rules engine is good at

Keep it. A rules engine is deterministic, instantly explainable, and hotfixable in minutes. For hard constraints — sanctions screening, hard velocity caps, known-bad BIN ranges, regulatory blocks — it's exactly the right tool, and no model should override it. Think of rules as your floor: the things that must always happen regardless of what any model thinks.

Where rules break down

Rules are axis-aligned splits that a human picked. That's the whole limitation:

Threshold brittleness. A hard cutoff at $500 means $499 sails through. Attackers measure this.
No interaction modeling. "New device" might be fine for a returning customer on their usual network and alarming for a first purchase shipping to a freight forwarder. A rule can't weigh the combination without you enumerating every case by hand.
Superlinear maintenance. Every rule interacts with every other rule. Past a few hundred, nobody can predict the effect of a change.
False positives on legitimate edge cases. The high-value transaction that looks exactly like the fraud pattern gets blocked. That specific failure — a rules-based engine with a high false-positive rate blocking legitimate high-value transactions — is what drove our fintech fraud-scoring engagement.

What gradient-boosted scoring changes

A boosted tree model learns feature interactions from labeled outcomes instead of from your guesses. You get a calibrated probability rather than a binary block, and you adapt by retraining rather than hand-editing logic. Concretely it changes:

From thresholds to gradients. The model uses amount relative to the entity's own history, in combination with everything else, instead of one global cutoff.
From "add a rule" to "retrain." New fraud patterns get learned from labels, not transcribed by an analyst.

The costs are real: you need labels (chargebacks, confirmed fraud) that arrive weeks late, a feature pipeline that computes the same values at training and serving time, and drift monitoring. And the model is no longer trivially explainable — you're now in post-hoc-attribution territory.

What an LLM layer changes, and what it doesn't

Be precise here, because this is where the hype is thickest. An LLM is not a better classifier for tabular transaction data. Gradient-boosted trees still win on structured features, at a fraction of the latency and cost. Where an LLM layer earns its place:

Reasoning over text. Merchant descriptors, order free-text, chargeback reason narratives, support history, device anomalies described in language — signals a tabular model can't ingest directly.
Readable rationales. A short natural-language explanation alongside the score, useful for analyst review — with the usual caveat that a fluent explanation still has to be checked for faithfulness.
Long-tail patterns with no labels. Using world knowledge to flag a descriptor pattern that's a known synthetic-identity tell, before you have the statistics to learn it.

What it doesn't change: it doesn't remove the boosted baseline, it adds latency and per-call cost, and its output has to be constrained and grounded or it becomes a hallucination risk on a money decision. On our fraud engagement the shape that worked was a gradient-boosted baseline plus an LLM reasoning layer over transaction narratives and device signals, with end-to-end scoring under 200ms — the LLM augments the score, it doesn't produce it.

The architecture that actually ships

Not either/or. Layered:

Rules for hard constraints and known-bad — instant accept/reject on the clear cases.
Boosted model for a calibrated risk score on everything in the grey zone.
LLM layer, selectively — only on borderline scores where narrative context or a readable rationale changes the outcome. Calling it on 100% of traffic is usually wasted latency and spend.
Policy layer on top: thresholds tuned to your loss trade-off, with human review for the highest-risk band.

Each layer does what it's best at. Rules give you a defensible floor and a fast hotfix path; the model handles interactions; the LLM handles language and explanation.

Where this stops being right

Low volume or low fraud rate. You may never accumulate enough labels to beat well-tuned rules plus manual review. Adding ML here is effort without payoff.
Sub-50ms or offline environments. An LLM call may be out of budget entirely; keep it to asynchronous review, or skip it.
Regulated declines. Every layer is another thing to explain and reconstruct. The LLM layer especially needs grounding and versioned logging, or it's a compliance liability rather than an asset.
Cost at scale. Per-transaction LLM inference is a real line item. Selective invocation isn't an optimization you add later — it's part of the design.

FAQ

Should we replace the rules engine entirely? No. Rules stay for hard constraints and instant hotfixes. The model takes over the grey zone where interactions matter.

Is an LLM accurate enough to score fraud on its own? Not on structured transaction data. It's an augmentation layer for text and rationale, sitting on top of a boosted classifier.

How much latency does the LLM layer add? Depends on model size and whether you call it on all traffic or only borderline cases. Selective invocation is how a real-time system stays real-time — our engagement held end-to-end scoring under 200ms with the layer in place.

ISTRALLEN builds layered fraud systems — rules, a model, and a reasoning layer — for fintech teams that need both accuracy and an auditable decision; see AI for Fintech.

See it in production

AI for Fintech → Fraud-scoring case study →

← All articles