In an optical display system having a waveguide and multiple diffractive optical elements (DOEs), an in-coupling DOE couples light into the waveguide, an intermediate DOE provides exit pupil expansion in a first direction, and an out-coupling DOE provides exit pupil expansion in a second direction and couples light out of the waveguide. The intermediate DOE includes grating features that are configured to rotate a state of polarization of light perturbations in the intermediate DOE that would otherwise generate optical interference through a closed-loop coupling phenomenon. The polarization state of a perturbed beam is rotated in the intermediate DOE to be orthogonal relative to the polarization state of the main beam used for image display to thereby uncouple the closed loops and make the perturbed beam non-interfering with the imaging beam.