A head-mounted device may have a head-mounted housing that is configured to be worn on a head of a user. While the head-mounted device is being worn, left and right displays in optical modules in the head-mounted device may provide images to eye boxes located rearward of the head-mounted device. A forward-facing publicly viewable display on a front portion of the head-mounted device may be covered with a transparent housing portion forming a display cover layer. Millimeter wave antennas may be mounted under a dielectric member that is interposed between the antennas and an edge portion of the transparent housing portion. The antennas may have planar outer surfaces. The dielectric member may have a planar surface separated from the planar outer surfaces by air gaps and a curved outer surface.