A disclosed structure includes a bipolar junction transistor (BJT) and a method of forming the structure. The structure includes a semiconductor layer on an insulator layer. The BJT includes a base region positioned laterally between emitter and collector regions. The emitter region includes an emitter portion of the semiconductor layer and an emitter semiconductor layer, which is within an emitter cavity in the insulator layer, which extends through an emitter opening in the emitter portion, and which covers the top of the emitter portion. The collector region includes a collector portion of the semiconductor layer and a collector semiconductor layer, which is within a collector cavity in the insulator layer, which extends through a collector opening in the collector portion, and which covers the top of the collector portion. Optionally, the structure also includes air pockets within the emitter and collector cavities.