Structures including a vertical heterojunction bipolar transistor and methods of forming a structure including a vertical heterojunction bipolar transistor. The structure comprises a semiconductor substrate including a trench, a first semiconductor layer including a portion adjacent to the trench, a dielectric layer between the first semiconductor layer and the semiconductor substrate, and a second semiconductor layer in the trench. The dielectric layer has an interface with the first semiconductor layer, and the second semiconductor layer includes a portion that is recessed relative to the interface. The structure further comprises a vertical heterojunction bipolar transistor including a collector in the portion of the second semiconductor layer.