A vehicle may have optical structures such as windows and mirrors that have the potential to allow glare from external objects to shine into the eyes of a driver or other vehicle occupant. A control circuit may gather information on where the eyes of the driver are located using a camera mounted in the vehicle and may gather information on where the sun or other source of glare are located outside of the vehicle. Based on this information, the control circuit may direct a light modulator on a window or mirror to selectively darken an area that prevents the glare from reaching the eyes of the driver. The light modulator may have a photochromic layer that is adjusted by shining light onto the photochromic layer, may be a liquid crystal modulator, an electrochromic modulator, or other light modulator layer.