A system that reduces the size of a design data set. During this design data set reduction operation, the system computes a decision boundary which separates a first group of data patterns in a training data set from a second group of data patterns in the training data set. For each data pattern in the training data set, the system determines if removing the data pattern from the training data set substantially affects the resulting decision boundary. If so, the system marks the data pattern as a key pattern. The system then removes all data patterns that are not marked as key patterns to produce a reduced training data set which represents the decision boundary.