A pattern inspecting technique, depending on the kind of materials, can reduce damage including shrinkage to materials when the materials are prone to such damage as shrinkage and spoilage caused by electron beam irradiation. This is accomplished by scanning a sample with a primary electron beam, detecting secondary electrons generated, or electrons reflected from the semiconductor device, or both the former and latter electrons, and converting the electrons into signals, and transforming the signals into an image, displaying the image, and detecting defective spots in the circuit pattern of the sample. The irradiation density (dose per unit area) of the electron beam is monitored and limited depending on the kind of material of the circuit pattern under inspection and the inspecting conditions, and damage, such as shrinkage and spoilage to the materials during electron beam irradiation, is reduced to an allowable range.