The present invention provides an inspection technique using a charged particle beam by which a method of setting a condition for optimally charging an object to be inspected without relying on an operator's experience is established and a voltage contrast image with higher efficiency of defect detection than ever before can be obtained. The inspection method comprises the steps of scanning an area on a surface of a substrate having a specific pattern formed thereon with a primary charged particle beam, detecting signals of secondary electrons emitted from the area, forming an image of the area from detected signals, and generating a histogram from the image. All these steps are performed each time a condition of irradiation with the charged particle beam is changed. When two or more separate peaks appear in the histogram, the histogram is determined as an optimal condition for inspection, and inspection is performed based on the image obtained under that condition.