An electronic device may be provided with a conductive sidewall and a phased antenna array having a dielectric resonator antenna aligned with an aperture in the conductive sidewall. A feed probe may excite the antenna to radiate through the aperture at a frequency greater than 10 GHz. The antenna may include an injection-molded plastic substrate that affixes the antenna to the peripheral conductive housing structures, thereby integrating the antenna into the conductive sidewall. A hole or other machining operation may be used to expose the feed probe through the injection-molded plastic substrate. Conductive interconnect structures may be inserted into the substrate and coupled to the feed probe. The interconnect structures may be soldered to a circuit board. The circuit board may be coupled to the feed probe through the interconnect structures. The circuit board may be mounted to a rear surface or a side surface of the injection-molded plastic substrate.