Gray level histograms for a test image and a reference image are adjusted by histogram scaling. Parameters from the histogram scaling are applied to the test image and the reference image. After the parameters are applied, the reference image and the test image are compared to produce a difference image, such as by subtracting the reference image from the test image. Noise in the difference image can be reduced, which improves defect identification in the difference image. In addition, noisy structures in the difference image which are elongated in vertical or horizontal direction can be found. If the noise exceeds a certain threshold, the structures may not be inspected.