Uplift Modeling — Marketing Incrementality
Overview
An applied causal inference and machine learning project using the benchmark Hillstrom dataset (MineThatData, 64,000 customers with randomized treatment) to optimize marketing spend. Rather than predicting propensity to purchase (which wastes money on organic buyers), uplift modeling estimates the Individual Treatment Effect (ITE) to isolate true incremental buyers.
Key Features
- Causal Meta-Learners: Implemented and evaluated Single-Learner (S-Learner), Two-Learner (T-Learner), and Class Transformation causal modeling approaches.
- Cloud Training on AWS SageMaker: Orchestrated model training jobs and automated Bayesian hyperparameter tuning using Amazon SageMaker Training Jobs and HyperparameterTuner.
- Causal Metric Evaluation: Audited model performance using Qini curves, Area Under the Uplift Curve (AUUC), and uplift@k across 10 repeated randomized splits.
- Customer Segmentation Matrix: Classified customer segments into Persuadables, Sure Things, Lost Causes, and Sleeping Dogs to maximize campaign ROI.
Architecture & Implementation
The pipeline ingests randomized trial data, standardizes customer recency, frequency, and monetary (RFM) attributes, and fits scikit-uplift models powered by Random Forest estimators on AWS SageMaker. The evaluation harness generates cumulative gain charts and Qini curves over repeated holdout folds.
Results & Performance
The A/B baseline proved the email campaign generated +6.1 percentage points incremental visit lift (+57% relative improvement, 16.7% vs 10.6%). In practical terms, the campaign produced approximately 6 additional visits per 100 contacted customers, allowing targeting algorithms to direct marketing budget exclusively to customers who alter their behavior because of the email.