Systems and methods for detecting defects are disclosed. According to certain embodiments, a method of performing image processing includes acquiring one or more images of a sample, performing first image analysis on the one or more images, identifying a plurality of first features in the one or more images, determining pattern data corresponding to the plurality of first features, selecting at least one of the plurality of first features based on the pattern data, and performing second image analysis of the at least one of the plurality of first features. Methods may also include determining defect probability of the plurality of first features based on the pattern data. Selecting the at least one of the plurality of first features may be based on the defect probability.