Methods and systems for classifying defects detected on a specimen with an adaptive automatic defect classifier are provided. One method includes creating a defect classifier based on classifications received from a user for different groups of defects in first lot results and a training set of defects that includes all the defects in the first lot results. The first and additional lot results are combined to create cumulative lot results. Defects in the cumulative lot results are classified with the created defect classifier. If any of the defects are classified with a confidence below a threshold, the defect classifier is modified based on a modified training set that includes the low confidence classified defects and classifications for these defects received from a user. The modified defect classifier is then used to classify defects in additional cumulative lot results.