Enhanced defect detection of a sample includes acquiring two or more inspection images from a sample from two or more locations of the sample for a first optical mode. The defect detection also generates an aggregated defect profile based on the two or more inspection images from the two or more locations for the first optical mode for a selected defect type and calculating one or more noise correlation characteristics of the two or more inspection images acquired from the two or more locations for the first optical mode. Defect detection further includes the generation of a matched filter for the first optical mode based on the generated aggregated defect profile and the calculated one or more noise correlation characteristics.