All light polarized in random directions is converted into s-polarized light and emitted by a polarizing conversion element 140. Linearly polarized light to be modulated is made incident on three liquid crystal light valves 250, 252, and 254, and p-polarized green light and s-polarized red and blue light are made incident on a cross-dichroic prism 260. Accordingly, for example, .lambda./2 phase plates 310 and 312 are respectively located in the optical paths before the first and third liquid crystal light valves 250 and 254. The .lambda./2 phase plates may be placed at various positions in the optical paths between dichroic mirrors 210 and 212 and the cross-dichroic prism 260.