Electronic devices such as head-mounted electronic devices may include displays for presenting images to users. To accommodate variations in the interpupillary distances associated with different users, a head-mounted device may have left-eye and right-eye optical modules that move with respect to each other. To hide internal structures from view, the rear of a head-mounted device may be provided with a cover. The cover may have a stretchable layer that is coupled to a frame. Openings in the stretchable layer may be aligned with the optical modules.