Structures and formation methods of a semiconductor device structure are provided. The semiconductor device structure includes a semiconductor substrate and a gate stack over the semiconductor substrate. The gate stack includes a gate dielectric layer and a work function layer. The gate dielectric layer is between the semiconductor substrate and the work function layer. The semiconductor device structure also includes a halogen source layer. The gate dielectric layer is between the semiconductor substrate and the halogen source layer.