Embodiments of a feature generation system and process for use in machine learning applications utilizing statistical modeling systems are described. In one embodiment, the feature generation process generates large feature spaces by combining features using logical, arithmetic and/or functional operations. A first set of features in an initial feature space are defined. Some or all of the first set of features are processed using one or more arithmetic, logic, user-defined combinatorial processes, or combinations thereof, to produce additional features. The additional features and at least some of the first set of features are combined to produce an expanded feature space. The expanded feature space is processed through a feature selection and optimization process to produce a model in a statistical modeling system.