A dimension measuring apparatus includes a light beam splitting element for splitting light emitted from a white light source into measuring light flux and reference light flux, a reference light scanning optics for varying optical path length of the reference light flux, a detector for detecting interference signal produced by the light fluxes, and a controller for determining the surface height of the object from the optical path length of the reference light flux corresponding to maximum value of the interference signal. The reference light scanning optics includes a rotary member, first and second reflective elements disposed to be symmetrical with respect to the rotation axis of the rotary member, and light beam deflecting members that direct the reference light flux to be incident on the first reflective element along the direction parallel and opposite to the incident direction of the reference light flux on the second reflective element.