Systems and methods for detecting defects on a specimen based on structural information are provided. One system includes one or more computer subsystems configured for separating the output generated by a detector of an inspection subsystem in an array area on a specimen into at least first and second segments of the output based on characteristic(s) of structure(s) in the array area such that the output in different segments has been generated in different locations in the array area in which the structure(s) having different values of the characteristic(s) are formed. The computer subsystem(s) are also configured for detecting defects on the specimen by applying one or more defect detection methods to the output based on whether the output is in the first segment or the second segment.