A method includes forming a gate dielectric layer on a semiconductor region, and depositing a first aluminum-containing work function layer using a first aluminum-containing precursor. The first aluminum-containing work function layer is over the gate dielectric layer. A second aluminum-containing work function layer is deposited using a second aluminum-containing precursor, which is different from the first aluminum-containing precursor. The second aluminum-containing work function layer is deposited over the first aluminum-containing work function layer. A conductive region is formed over the second aluminum-containing work function layer.