An electronic device may be provided with a display mounted in a housing. Color ambient light sensors may make measurements of ambient light intensity and color through windows in an inactive border region of the display or other portions of the device. The electronic device may process the ambient light measurements based on ambient light information from the ambient light sensors and based on information from additional sensors such as an image sensor, a force sensor, a capacitive touch sensor, a proximity sensor, an orientation sensor, and other devices. Control circuitry in the electronic device may produce reliable ambient light measurements by combining readings from multiple reliable sources and by discarding readings from ambient light sensors that are blocked by a user's fingers or other external objects. Display color cast and intensity may be adjusted based on ambient light information.