A gate structure is formed straddling a first portion of a plurality of semiconductor fins that extend upwards from a topmost surface of an insulator layer. A dielectric spacer is formed on sidewalls of the gate structure and straddling a second portion of the plurality of semiconductor fins. Epitaxial semiconductor material portions that include a non-planar bottommost surface and a non-planar topmost surface are grown from at least the exposed sidewalls of each semiconductor fin not including the gate structure or the gate spacer to merge adjacent semiconductor fins. A gap is present beneath epitaxial semiconductor material portions and the topmost surface of the insulator layer. A second epitaxial semiconductor material is formed on the epitaxial semiconductor material portions and thereafter the second epitaxial semiconductor material is converted into a metal semiconductor alloy.