A system and method for training an SVM in a scalable manner includes boosting the SVM during training. Specifically, individual SVMs in an ensemble of SVMs are trained using small subsets of a training data set, with data that earlier classifiers in the ensemble incorrectly classified being overrepresented in succeeding subsets. In this way, the speed with which the overall SVM is trained is increased and the memory requirements therefor are reduced, even for relatively large training data sets.