Book a call
HomeServicesPortfolioAboutContactBlogCareers
Book a call
Fintech

How Real-Time Fraud Scoring Systems Are Architected

August 2026 · ISTRALLEN Team

The requirement that turns into an architecture problem

Score every transaction for fraud before it completes. Simple sentence, then you look at the budget: a few hundred milliseconds end to end — feature lookup, model inference, policy evaluation, logging — at peak throughput, with nothing dropped when a downstream service goes slow. That budget is why real-time fraud scoring architecture is an infrastructure problem first and a modeling problem second.

transactioncheckout, $ amountpeak throughputfeature retrievalvelocity · entity · deviceoften eats half the budgetthe usual bottleneckmodel inferenceboosted tree — ms scale+ LLM, grey zone only,timeout → boosted fallbackpolicy + responsethreshold checkaccept · decline · reviewwithin the 200ms budget~200ms synchronous budgetstale-feature OKfeatures up to Ns oldflagged as staleboosted-only pathLLM timeout →return boosted scoreconservative defaultmodel service down →rules + tighter thresholdasync: audit + retrainfull snapshot, versions, loggedchargebacks join back, weeks lateroff the hot path — never blocks
Everything above the dashed line runs on the clock: feature retrieval, model inference, and policy evaluation all compete for the same roughly 200-millisecond budget, and a slow feature store usually eats more of it than the model itself. Two of the three fallbacks shown below don't belong to policy at all — they're both failure modes of model inference: an optional LLM call that times out falls back to the boosted score alone, while the model service being unreachable entirely falls back to rules and a tighter threshold. Only the feature store gets its own fallback, accepting stale values rather than blocking on a fresh read. Nothing about the response waits on what happens next: the full snapshot, model and policy versions, and any LLM rationale are logged asynchronously, off the hot path, and chargeback labels arriving weeks later close the loop with a dashed line back into model inference — retraining as a standing pipeline, not a one-time run.

The latency budget

Everything on the synchronous path competes for the same ~200ms. Roughly where it goes:

  • Network + auth handshake — fixed overhead you don't control.
  • Feature retrieval — the usual bottleneck. Velocity aggregates, entity lookups, device history.
  • Model inference — cheap for boosted trees (single-digit ms), more for anything neural or an LLM call.
  • Policy / threshold evaluation — fast, but it's still in the budget.
  • Response — return the decision.
  • Logging — must be asynchronous; more on that below.

If feature retrieval takes 120ms, you've spent more than half your budget before the model runs. That's why the feature store dominates the design.

The feature store is the hard part

Real-time features — counts and sums over the last 1m / 1h / 24h per card, device, account, IP; entity-level aggregates; "seen before" flags — have to be precomputed and read in single-digit milliseconds. The usual shape:

  • A streaming pipeline (Kafka or equivalent) consuming the transaction event stream and updating an online store keyed by entity.
  • The same feature definitions computed in batch over historical data for training, so the model sees identical semantics in training and serving. Train/serve skew here is a silent killer — the model learns on one definition of "24h velocity" and scores on another.
  • Point-in-time correctness when building training sets: each label joins to feature values as they were at that moment, not as they are now.

Most of the engineering time on a real-time fraud system goes here, not into the model.

Model serving

Boosted tree models evaluate in milliseconds; the work is assembling the feature vector and versioning. Keep the model behind a thin service that stamps the model version into every response and every log line. When you retrain, you need to know exactly which version scored a given transaction — the compliance reviewer six weeks later depends on it, and so does any honest post-mortem.

If there's an LLM reasoning layer, it does not sit inline for 100% of traffic. Call it only on grey-zone scores, with a strict timeout and a fallback to the boosted score if it doesn't answer in budget. On our fintech fraud-scoring project this selective pattern is how a boosted baseline plus an LLM layer over transaction narratives and device signals still scored end to end in under 200ms.

Graceful degradation

Plan explicitly for the feature store being slow or the LLM timing out. The system needs named fallbacks:

  • Stale-feature tolerance — accept features up to N seconds old, flagged as stale, rather than blocking on a fresh read.
  • Boosted-only path — if the LLM layer doesn't respond, return the model score.
  • Conservative default policy — if the model service itself is unreachable, fall back to rules and a tighter threshold.

A fraud system that fails open leaks money; one that fails closed blocks good customers during an incident. Neither is universally right — choose per failure mode, deliberately, and write it down.

The async path

Everything not required to return the decision goes off the hot path: the full feature snapshot, attribution values, model version, policy version, any LLM rationale — appended to an immutable store for audit and future training. This append-only record is what makes a decision reconstructable months later, which on a regulated engagement is a hard requirement, not a nice-to-have.

The feedback loop

Chargebacks and confirmed-fraud labels arrive days to weeks after the decision. You need a pipeline that joins them back to the logged decisions, tracks score calibration and feature drift, and retrains on a schedule. Skip this and the model rots quietly — the metrics look fine because you're grading against stale assumptions, while the fraud mix has moved on.

Where this stops being right

  • You may not need synchronous blocking. If you can review and claw back — some payment flows, most marketplace payouts — an asynchronous scoring pipeline is dramatically cheaper and simpler. Don't build real-time you won't use.
  • Very low volume. A managed fraud API, or rules plus manual review, beats standing up a streaming stack and an online feature store for a few thousand transactions a day.
  • Large grey zone + inline LLM. If a big fraction of traffic lands borderline, even selective LLM invocation adds up in latency and cost. Tighten the band or budget for it.
  • Multi-region / data residency. Replicating an online feature store across regions with acceptable consistency is its own project — scope it separately.

FAQ

What's a realistic latency target? Sub-200 to 300ms end to end is typical for inline scoring. Our fintech engagement held scoring under 200ms with a boosted baseline and a selective LLM layer.

Do we actually need Kafka? You need a streaming path that keeps real-time aggregates current. The specific technology matters far less than train/serve consistency and point-in-time correctness.

Where does the LLM layer go? Off the 100%-traffic hot path. Borderline scores only, strict timeout, automatic fallback to the boosted score.

ISTRALLEN designs real-time fraud scoring for fintech teams — feature store, serving, degradation, and audit trail as one system; details under AI for Fintech.

See it in production
AI for Fintech → Fraud-scoring case study →
← All articles