A plurality of semiconductor fins are formed which extend from a semiconductor material portion that is present atop an insulator layer of a semiconductor-on-insulator substrate. A gate structure and adjacent gate spacers are formed that straddle each semiconductor fin. Portions of each semiconductor fin are left exposed. The exposed portions of the semiconductor fins are then merged by forming an epitaxial semiconductor material from an exposed semiconductor material portion that is not covered by the gate structure and gate spacers.