Provided is an optical image measuring apparatus capable of obtaining a high-accuracy image without being influenced by a movement of an object to be measured. Flash light is emitted from a xenon lamp (2) and converted into broad band light by an optical filter (2A). A polarization characteristic of the flash light is converted into linear polarization by a polarizing plate (3). Then, the flash light is divided into signal light (S) and reference light (R) by a half mirror (6). A polarization characteristic of the reference light (R) is converted into circular polarization by a wavelength plate (7). The signal light (S) and the reference light (R) are superimposed on each other by the half mirror (6) to produce interference light (L). A CCD (23) detects interference light having the same characteristic as that of the produced interference light (L). The produced interference light (L) is divided into an S-polarized light component (L1) and a P-polarized light component (L2) by a polarization beam splitter (11). The polarized light components are detected by CCDs (21 and 22). A signal processing section (20) of a computer (30) forms an image of the object to be measured (O) based on detection signals from the CCDs (21, 22, and 23).