System Design Lab
Real-time fraud detection is a tight synchronous latency budget wrapped around streaming features and a model.
Tune transaction rate, the decision latency budget, the feature window, model inference time, entities tracked, and feedback delay. The design evolves from static rules, to batch ML scoring, to streaming features with synchronous scoring, to a feedback loop with retraining, and finally to graph features at scale.
Guided walkthrough
Reason about it one step at a time
Takeaway
Normal evolution scenarios
Click left to right for the intended demo path. Each card changes the workload inputs.
Recommended shape
Current architecture path
Clients
Payment client submits each transaction and waits for an allow, deny, or review decision
Scoring API
Scoring API orchestrates the synchronous decision within the latency budget
Decision gate applies fail-open or fail-closed policy and returns the verdict
Real-time features
Stream processor maintains windowed velocity counts and aggregates from the event stream
Feature store serves precomputed per-entity features at low latency to the scorer
Model + rules
Rules engine evaluates deterministic rules and hard blocks before or beside the model
Model service returns a fraud probability the gate combines with the rules verdict
Graph store links shared cards, devices, and addresses to expose coordinated rings
Feedback + retrain
Label store collects chargebacks and dispute outcomes as ground-truth labels
Training pipeline retrains on fresh labels to track drifting fraud patterns and ships new models
Bottlenecks
Scoring path load
Decision budget headroom
Stream state size
Graph linking load
Model drift risk
Why this changes
Decision tradeoffs
Source-backed rules
These are the durable system-design claims behind the model. The exact slider thresholds are deliberately labeled as teaching assumptions.
Stateful stream processing maintains windowed aggregates in real time
Flink keeps keyed state and windows per entity so velocity counts and aggregates update continuously instead of being recomputed per request.
Apache Flink DocsA durable, replayable log backs the feature and feedback streams
Kafka gives the transaction and label streams an ordered, replayable backbone, which is what lets features be recomputed and models retrained from history.
Apache Kafka DocsA feature store serves the same features online at low latency and offline for training
A feature store solves online/offline skew: the synchronous scorer reads fresh features fast while training reads consistent historical values.
Feast DocsFraud detection is an anomaly / novelty-detection problem with drift
Fraud is rare and the patterns shift over time, so detection is framed as outlier/novelty detection that needs fresh labels and frequent retraining rather than one static model.
scikit-learnTeaching assumptions
- The decision budget is split across feature lookup, model inference, and rules; remaining headroom is what is left after model latency.
- Single-node scoring, feature-lookup, and stream-state budgets are conservative teaching numbers, not vendor limits.
- Graph linking cost is modeled as growing with both entities tracked and transaction rate; real systems precompute much of it offline.