After formation of gate structures over semiconductor fins and prior to formation of raised active regions, a directional ion beam is employed to form a dielectric material portion on end walls of semiconductor fins that are perpendicular to the lengthwise direction of the semiconductor fins. The angle of the directional ion beam is selected to be with a vertical plane including the lengthwise direction of the semiconductor fins, thereby avoiding formation of the dielectric material portion on lengthwise sidewalls of the semiconductor fins. Selective epitaxy of semiconductor material is performed to grow raised active regions from sidewall surfaces of the semiconductor fins. Optionally, horizontal portions of the dielectric material portion may be removed prior to the selective epitaxy process. Further, the dielectric material portion may optionally be removed after the selective epitaxy process.