A pattern inspection apparatus determines a difference of the measured dislocation of respective alignment marks of an opaque pattern and a phase shifting pattern (measurement difference), in addition to a difference between the both alignment mark positions in design (design difference). A difference between the measurement difference and the design difference is set as a difference in alignment mark position between the opaque pattern and the phase shifting pattern in a reference pattern which is later used in inspection. In this manner, by correcting one pattern data with respect to the other pattern data in the reference pattern, the displacement generated in the both patterns can be reflected, and the reference pattern data regarding an image of a sample which is actually observed can be created.