Defect classification includes acquiring one or more images of a specimen including multiple defects, grouping the defects into groups of defect types based on the attributes of the defects, receiving a signal from a user interface device indicative of a first manual classification of a selected number of defects from the groups, generating a classifier based on the first manual classification and the attributes of the defects, classifying, with the classifier, one or more defects not manually classified by the manual classification, identifying the defects classified by the classifier having the lowest confidence level, receiving a signal from the user interface device indicative of an additional manual classification of the defects having the lowest confidence level, determining whether the additional manual classification identifies one or more additional defect types not identified in the first manual classification, and iterating the procedure until no new defect types are found.