A structure includes a semiconductor substrate including a first semiconductor region and a second semiconductor region, a first transistor in the first semiconductor region, and a second transistor in the second semiconductor region. The first transistor includes a first gate dielectric over the first semiconductor region, a first work function layer over and contacting the first gate dielectric, and a first conductive region over the first work function layer. The second transistor includes a second gate dielectric over the second semiconductor region, a second work function layer over and contacting the second gate dielectric, wherein the first work function layer and the second work function layer have different work functions, and a second conductive region over the second work function layer.