Disclosed is a method of inspecting a defect present at a surface portion of a photomask blank which includes an optical film, and a thin film. The method includes: selecting and designating an inspection treatment procedure and a criterion for determination of rugged shape of the defect which correspond to modes of the optical film and the thin film of the photomask blank; applying inspection light to a region including the defect while maintaining a distance between the defect and an objective lens of an inspecting optical system, based on the designated inspection treatment procedure, and collecting reflected light from the region irradiated with the inspection light, as a magnified image of the region, through the inspecting optical system; and determining the rugged shape of the defect, from light intensity distribution of the magnified image, based on the designated criterion for determination.