A pattern defect inspection apparatus capable of detecting minute defects on a sample with high sensitivity without generating speckle noise in signals is realized. Substantially the same region on a surface of a wafer is detected by using two detectors at mutually different timings. Output signals from the two detectors are summed and averaged to eliminate noise. Since a large number of rays of illumination light are not simultaneously irradiated to the same region on the wafer, a pattern defect inspection apparatus capable of suppressing noise resulting from interference of a large number of rays, eliminating noise owing to other causes and detecting with high sensitivity minute defects on the sample without the occurrence of speckle noise in the signal can be accomplished.