A semiconductor wafer 11 is irradiated for scanning with a charged particle beam 6 so as to detect secondary charged particles 9 obtained from the wafer 11 as a result of the irradiation of the beam 6. A detected image of an inspection area obtained based on scanning information and on a detection signal derived from the secondary charged particles 9 is compared with a detected image of a reference area to find a difference therebetween. The difference is compared with a threshold value to detect a defect candidate. Defect information including positional information about the defect candidate is generated in such a manner as to include a relative position of a predetermined feature point within each of repeat patterns formed on the semiconductor wafer 11 with regard to the origin of a coordinate area established in each of these repeat patterns, and a relative position of the defect candidate with regard to the feature point. This contributes to providing a defect inspecting apparatus capable of determining defective areas for extraction by FIB more easily than before.