After forming a plurality of first semiconductor fins having a first spacing in a logic device region and a plurality of second semiconductor fins having a second spacing in a memory device region, sacrificial spacers are formed on sidewalls of the plurality of the first semiconductor fins and the plurality of the second semiconductor fins to completely fill spaces between the plurality of first semiconductor fins, but only partially fill spaces between second semiconductor fins. Next, dielectric barrier layer portions are formed in gaps between the sacrificial spacers. After removal of the sacrificial spacers, an entirety of the plurality of first semiconductor fins is laterally enclosed by a corresponding pair of neighboring dielectric barrier layers, while each of the plurality of second semiconductor fins is laterally enclosed by a corresponding pair of neighboring dielectric barrier layer portions.