In a replacement gate scheme, after formation of a gate dielectric layer, a work function material layer completely fills a narrow gate trench, while not filling a wide gate trench. A dielectric material layer is deposited and planarized over the work function material layer, and is subsequently recessed to form a dielectric material portion overlying a horizontal portion of the work function material layer within the wide gate trench. The work function material layer is recessed employing the dielectric material portion as a part of an etch mask to form work function material portions. A conductive material is deposited and planarized to form gate conductor portions, and a dielectric material is deposited and planarized to form gate cap dielectrics.