Systems and methods provide for feature selection that combines semantic classification and generative filtering with forward selection. Features from an original feature set are divided into feature subsets corresponding to ranked semantic classes. Additionally, low quality features are removed from consideration. Features are selected for a reduced feature set by iteratively processing the feature subsets using forward selection in an order corresponding to the ranking of the semantic classes. The reduced feature set is used to generate a predictive model.