A system for detecting abnormal patterns in a surface pattern on the surface of a sample wherein, when the abnormal pattern has a portion smaller than the surface pattern, the light for illuminating the sample is stopped down to an opening so that the abnormal pattern can be distinguished over the surface pattern, the surface pattern is limited to a predetermined opening by the illumination light to form an image on a detector, and the surface pattern detected by the detector is processed to detect the abnormal pattern in the surface pattern. Further, the plurality of light rays having good directivity are illuminated onto a point on the surface of the sample from dissimilar directions maintaining a predetermined angle of incidence, the detect signals are separated to be corresponded to said light rays, and the above processing is carried out to enhance the precision of detection.