An incremental greedy method to feature selection is described. This method results in a final classifier that performs optimally and depends on only a few features. Generally, a small number of features is desired because it is often the case that the complexity of a classification method depends on the number of features. It is very well known that a large number of features may lead to overfitting on the training set, which then leads to a poor generalization performance in new and unseen data. The incremental greedy method is based on feature selection of a limited subset of features from the feature space. By providing low feature dependency, the incremental greedy method 100 requires fewer computations as compared to a feature extraction approach, such as principal component analysis.