A head-mounted device may have left and right optical modules that present images to a user's eyes. Each optical module may have an optical module support structure and a lens and display coupled to the optical module support structure. The head-mounted device may have a head-mounted housing that supports the optical modules. A cover on a rear face of the head-mounted housing may have a pair of openings configured to receive the left and right optical modules. The cover may have a cover layer and left and right cover layer mounting rings respectively configured to engage with the optical module support structures of the left and right optical modules. Sets of magnets in the left and right optical modules may be configured to attract corresponding left and right vision correction lenses. Gaskets may surround the lenses in the optical modules and may prevent environmental contaminant intrusion.