A method includes forming isolation regions extending into a semiconductor substrate, wherein semiconductor strips are located between the isolation regions, and forming a dielectric dummy strip between the isolation regions, recessing the isolation regions. Some portions of the semiconductor strips protrude higher than top surfaces of the recessed isolation regions to form protruding semiconductor fins, and a portion of the dielectric dummy strip protrudes higher than the top surfaces of the recessed isolation regions to form a dielectric dummy fin. The method further includes etching the dielectric dummy fin so that a top width of the dielectric dummy fin is smaller than a bottom width of the dielectric dummy fin. A gate stack is formed on top surfaces and sidewalls of the protruding semiconductor fins and the dielectric dummy fin.