A structure includes a plurality of semiconductor regions, a first gate stack and a second gate stack immediately neighboring each other, a first fin isolation region in the first gate stack, and a second fin isolation region in the second gate stack. The first fin isolation region and the second fin isolation region have a sideway overlap having an overlap distance being equal to or greater than a pitch of the plurality of semiconductor regions. The overlap distance is measured in a direction parallel to lengthwise directions of the first gate stack and the second gate stack. A plurality of source/drain regions are on opposing sides of the first gate stack and the second gate stack to form a plurality of transistors.