An optical image measuring apparatus capable of speedily starting image measurement in response to a measurement start request and performing image measurement on a region on which focus is achieved at determination of an image region. When observation light (M) is focused on an observation CCD camera (23), measurement CCD cameras (21, 22) are disposed to have a measurement object point (Q) in a position shifted from point (P) to an apparatus side. In response to a measurement start switch (60), an interferometer moving stage (50) is moved to adjust an optical path length difference, thereby shifting the point (Q) to the point (P). The distance (d) is set to equal to or longer than necessary to move the interferometer moving stage (50) which accelerates a moving speed thereof to a predetermined speed. Thereby, image measurement can speedily start in response to request for image measurement of an object (O). It is possible to perform image measurement on the region where focus is achieved at the time of determination of the measurement region, i.e., at the position of the point (P).