Structures for a bipolar junction transistor and methods of forming a structure for a bipolar junction transistor. The structure includes a dielectric layer having a cavity, a first semiconductor layer on the dielectric layer, a collector including a portion on the first semiconductor layer, an emitter including a portion on the first semiconductor layer, and a second semiconductor layer that includes a first section in the cavity and a second section. The second section of the second semiconductor layer is laterally positioned between the portion of the collector and the portion of the emitter.