Financial Fraud Detection Pipeline with Amazon SageMaker and Control-M
BMC developed an automated, scalable machine learning pipeline to detect payment fraud with high accuracy and minimal false positives in the financial services sector. The solution uses AWS technologies including Amazon SageMaker, Amazon Redshift, Amazon EKS, Amazon Athena, AWS Lambda, and is orchestrated with Control-M for seamless automation and workflow management. The pipeline automates data extraction from Redshift to S3, preprocessing via Kubernetes on Amazon EKS, model training and evaluation with Amazon SageMaker, and visualization using Athena and Amazon QuickSight. Three ML models (logistic regression, decision tree classifier, multi-layer perceptron) are trained and evaluated to select the best performing one for production. Control-M provides orchestration, scheduling, job dependency management, and automated execution to reduce manual intervention and improve operational efficiency.
- Organization
- BMC
- Industry
- Finance
- Location
- United States
- Published
- March 2025
Reported outcomes
Strategic outcomes
Primary read
Use case focus
Showing 3 of 3
- 1Fraud Detection
- 2Machine Learning Pipeline
- 3Workflow Automation
- Developing a robust fraud detection system to handle high transaction volumes and evolving fraud tactics.
- Ensuring high accuracy with minimal false positives in payment fraud detection.
- Building an automated, scalable, and efficient ML pipeline capable of continuous model training, evaluation, and monitoring.
- Leveraged AWS managed services including Amazon SageMaker for ML training, Amazon Redshift for data warehousing, Amazon EKS for containerized preprocessing, Amazon Athena for querying, and AWS Lambda for data validation and Athena integration.
- Used Control-M integration plug-ins to orchestrate the entire workflow from data extraction, validation, preprocessing, model training, evaluation, and visualization pipelines.
- Implemented Kubernetes jobs to preprocess raw financial data using containerized Python scripts.
- Automated model training and evaluation in SageMaker pipelines for logistic regression, decision tree, and MLP models.
- Scheduled monitoring and results visualization through Athena and Amazon QuickSight, enabling real-time insights.
- Improved fraud detection accuracy, with the decision tree classifier showing the most balanced performance metrics.
- Significant reduction in operational overhead due to automated pipeline orchestration with Control-M.
- Enhanced real-time fraud detection efficiency through automated, end-to-end workflow management.
- Facilitated continuous training and evaluation adapting to evolving fraud patterns in financial transactions.
- Increased ability for financial institutions to detect fraud proactively, reducing financial losses and enhancing customer trust.
Architecture
The pipeline orchestrates data extraction from Amazon Redshift to S3, data validation with AWS Lambda, preprocessing on Amazon EKS Kubernetes cluster with containerized Python scripts, followed by automated training and evaluation in Amazon SageMaker. The pipeline outputs evaluation metrics loaded into Amazon Athena and visualized with Amazon QuickSight. Control-M manages the workflow orchestration including scheduling, event triggers, and dependency management.
Implementation partners1
Sources & evidence1
AI-generated summary. Verify important details with the linked sources before relying on this case.
Explore related AI use cases
Was this useful?