Methods and systems for detecting defects on a specimen are provided. One system includes one or more computer subsystems configured for acquiring images generated by an imaging subsystem at multiple instances of a pattern of interest (POI) within a die formed on the specimen. The multiple instances include two or more instances that are located at aperiodic locations within the die. The computer subsystem(s) are also configured for generating a POI reference image from two or more of the images generated at the multiple instances of the POI within the die. The computer subsystem(s) are further configured for comparing the images generated at the multiple instances of the POI within the die to the POI reference image and detecting defects in the multiple instances of the POI based on results of the comparing.