A method of forming an optical element for a color display device includes forming a lens shaped die, coating a low refractive index photo-setting resin on the lens shaped die, pasting the lens shaped die to a substrate, and irradiating the coated low refractive index photo-setting resin so as to set the low refractive index resin and form a low refractive index layer. The lens shaped die is removed from the low refractive index layer set on the substrate, a high refractive index photo-setting resin is coated on the low refractive index layer by use of a planarizing die, and the coated high refractive index photo-setting resin is irradiated so as to set the high refractive index resin and form a high refractive index layer. The planarizing die is then from the high refractive index layer set on the low refractive index layer.