Real-Time Fraud Detection
A production-grade credit card fraud detection system with sub-15ms latency, online/offline feature store, and automated MLOps lifecycle.
Airflow login: viewer / viewer
Overview
A production-grade, real-time credit card fraud detection system designed to ingest transaction streams and classify fraud risk with sub-50ms latency. The system leverages an online/offline feature store to compute behavioral velocity features without training-serving skew, running a tuned gradient-boosted classifier behind a three-tier decision-action framework.
Performance Metrics
Key Features
Zero Training-Serving Skew
Feast standardizes feature generation between offline training (PostgreSQL) and online inference (Redis), ensuring consistency.
3-Tier Decision Framework
Instead of binary fraud/not-fraud, predictions map to actionable outcomes: Auto-Block, Manual Review, and Soft Audit Signal.
Automated MLOps Lifecycle
Airflow DAGs monitor data drift with Evidently AI and automatically trigger retraining, validation gating, and model registry promotion.
Resilient Streaming Ingestion
Kafka queues transaction bursts to buffer ingestion rate from scoring engines, enabling high-throughput processing.
Unified Observability
Real-time transaction WebSockets, audit logs, and performance metrics via Prometheus and Grafana dashboards.
Architecture
Ingestion
Raw transaction events streamed to Apache Kafka's fraud-transactions topic.
Processing & Enrichment
Python consumer pulls events and retrieves pre-computed velocities from Redis via Feast.
Inference
Tuned CatBoost model scores enriched features and maps to the 3-tier decision framework.
Action & Auditing
Decisions written to PostgreSQL audit log, streamed to React dashboard via WebSockets, emitted to Kafka fraud-decisions topic.
Feedback Loop
Airflow schedules daily drift detection (Evidently) and weekly retraining, promoting models to DAGsHub MLflow.
Dashboard
Live Transactions
Real-time WebSocket feed with fraud-rate chart and per-row tier badges.
Demo Results
Confusion matrix, F1 score, and tier breakdown from batch test runs.
Drift Report
Evidently AI drift detection with per-feature KS tests and Wasserstein distances.
Grafana Monitoring
Real-time system metrics — prediction latency, throughput, CPU/memory usage, and container health.
Airflow Pipelines
Orchestrated DAGs for drift detection, weekly retraining, and monthly hyperparameter tuning.
