An electronic device may have an optical system that includes one or more light-based components. The light-based components may include light-emitting components such as light-emitting diodes or lasers and may include light-detecting components such as photodiodes or digital image sensors. The optical system may include a light diffuser. The light diffuser may diffuse light that is being detected by a light-detecting component or may diffuse light that is being emitted by a light-emitting component. Light diffusers in optical systems may be formed from patterned light diffuser layers on transparent substrates. Layers of sealant, thin glass layers, antireflection coatings, and other layers may be incorporated into the light diffusers. The light diffuser layers may operate at visible wavelengths and infrared wavelengths. An infrared light diffuser layer may be formed from a patterned silicon layer such as a patterned layer of hydrogenated amorphous silicon.