Weak Learners, Sequential Correction, and AdaBoost
Explore the concept of weak learners and how AdaBoost sequentially corrects their errors to build strong predictive models. Understand how example weights focus training on misclassified data, enabling effective fraud detection. This lesson helps you grasp boosting fundamentals and distinguish AdaBoost from other ensemble methods, preparing you to apply these techniques in real-world scenarios.
A one-split fraud classifier might flag a transaction as likely chargeback fraud only when its amount exceeds $500. That rule can catch some fraudulent charges, but it misses fraud cases below $500 that have unusual locations or repeated payment attempts. Missing true fraud can leave a chargeback exposure unaddressed, while making the tree much deeper can create an overfitting risk.
A weak learner is a model that performs only slightly better than random guessing on the training task. A decision stump, a tree with one split, is a common weak learner. The stump has limited capacity, so it underfits when fraud depends on several small signals rather than one dominant rule.
Boosting combines many weak learners into one stronger model. Each weak learner contributes a small correction, so the ensemble can capture patterns that no single stump can represent. The goal is not to make one stump solve fraud detection. The goal is to make later stumps concentrate on the transactions that earlier stumps misclassified.
Sequential correction instead of independent voting
Unlike bagging, boosting does not train all base learners independently. Bagging creates varied training samples and aggregates their predictions, mainly to reduce variance. Boosting trains one learner after another, mainly to reduce bias by correcting earlier errors.
Consider eight chargeback ...