A three-dimensional shape measuring device includes: an interference objective lens including an interfering unit configured to separate part of measurement light as reference light and generate multiplexed light of the measurement light returning from the surface to be measured and the reference light returning from a reference surface, and an objective lens configured to cause the measurement light to focus on the surface to be measured; a scanning unit configured to cause the interference objective lens to scan relatively with respect to the surface to be measured; an image capturing unit configured to repeatedly capture an image of the multiplexed light and output a plurality of captured images during scanning; and a first signal processing unit configured to calculate the three-dimensional shape of the surface to be measured, based on a result of comparing the sharpness of each pixel on the same coordinate of the plurality of captured images.