A metrology system includes a radiation source that generates light, an optical modulation unit, a reflector, an interferometer, and a detector. The optical modulating unit temporally separates a first polarization mode of the light from a second polarization mode of the light. The reflector directs the light towards a substrate. The interferometer interferes the diffracted light from a pattern on the substrate, or reflected light from the substrate, and produces output light from the interference. The detector receives the output light from the interferometer. The first and second polarization modes of the output light are temporally separated at the detector.