A semiconductor device and a method of forming the same, the semiconductor device include a substrate, and a first gate structure and a second gate structure disposed on the substrate. The first gate structure includes a barrier layer, a first work function layer, a second work function layer and a conductive layer stacked one over another on the substrate. The second gate structure includes the barrier layer, a portion of the first work function layer and the conductive layer stacked one over another on the substrate, wherein the portion of the first work function layer has a smaller thickness than a thickness of the first work function layer.