A semiconductor device includes a semiconductor substrate; a plurality of channel regions, including a p-type channel region and an n-type channel region, disposed over the semiconductor substrate; and a gate structure. The gate structure includes a gate dielectric layer disposed over the plurality of channel regions and a work function metal (WFM) structure disposed over the gate dielectric layer. The WFM structure includes an n-type WFM layer over the n-type channel region and not over the p-type channel region and further includes a p-type WFM layer over both the n-type WFM layer and the p-type channel region. The gate structure further includes a fill metal layer disposed over the WFM structure and in direct contact with the p-type WFM layer.