A three-dimensional measurement device includes: an optical system that splits incident light into two lights and radiates lights to a measurement object and to a reference surface, and recombines the two lights to emit combined light; a first irradiator that emits first light including first polarized light and entering a first surface; a second irradiator that emits second light including second polarized light and entering a second surface; a first imaging system to which the first output light enters wherein the first output light is emitted from the first surface when the first light enters the first surface; a second imaging system to which the second output light enters wherein the second output light is emitted from the second surface when the second light enters the second surface; and an image processor that performs three-dimensional measurement based on interference fringe images obtained by the first and second imaging systems.