After forming an interlevel dielectric (ILD) layer over a semiconductor material portion located on a substrate, a gate trench is formed extending through the ILD layer to expose a channel region of the semiconductor material portion. A gate structure is then formed within the gate trench. Epitaxial semiconductor regions are subsequently formed within source/drain contact openings formed on opposite sides of the gate structure, followed by forming source/drain contact structures on the epitaxial semiconductor regions.