A bipolar transistor is supported by a single-crystal silicon substrate including a collector connection region. A first epitaxial region forms a collector region doped with a first conductivity type on the collector connection region. The collector region includes a counter-doped region of a second conductivity type. A second epitaxial region forms a base region of a second conductivity type on the first epitaxial region. Deposited semiconductor material forms an emitter region of the first conductivity type on the second epitaxial region. The collector region, base region and emitter region are located within an opening formed in a stack of insulating layers that includes a sacrificial layer. The sacrificial layer is selectively removed to expose a side wall of the base region. Epitaxial growth from the exposed sidewall forms a base contact region.