Learning Ensembles from Bites A Scalable and Accurate.pdf

Journal of Machine Learning Research 5 (2004) 421-451 Submitted 3/03; Revised 11/03; Published 4/04 Learning Ensembles from Bites: A Scalable and Accurate Approach Nitesh V. Chawla N ITE SH .CHAWLA @CIBC .CA Customer Behavior Analytics, CIBC Commerce Court East, 11th Floor Toronto, ON M5L 1A2, Canada Lawrence O. Hall HALL @C SEE .U SF.EDU Department of Computer Science and Engineering University of South Florida Tampa, FL 33620, USA Kevin W. Bowyer KWB @C SE .N D .EDU Department of Computer Science and Engineering University of Notre Dame 384 Fitzpatrick Hall Notre Dame, IN 46556, USA W. Philip Kegelmeyer WPK @CA .SAN DIA .GOV Sandia National Labs, Biosystems Research Department P.O. Box 969, MS 9951 Livermore, CA 94551-0969, USA Editor: Claude Sammut Abstract Bagging and boosting are two popular ensemble methods that typically achieve better accuracy than a single classifier. These techniques have limitations on massive data sets, because the size of the data set can be a bottleneck. Voting many classifiers built on small subsets of data (“pasting small votes”) is a promising approach for learning from massive data sets, one that can utilize the power of boosting and bagging. We propose a framework for building hundreds or thousands of such classifiers on small subsets of data in a distributed environment. Experiments show this approach is fast, accurate, and scalable. Keywords: ensembles, bagging, boosting, diversity, distributed learning