Structures and formation methods of a semiconductor device structure are provided. The semiconductor device structure includes a semiconductor substrate and a gate electrode over the semiconductor substrate. The semiconductor device structure also includes a source/drain structure adjacent to the gate electrode. The semiconductor device structure further includes a spacer element over a sidewall of the gate electrode, and the spacer element has an upper portion having a first exterior surface and a lower portion having a second exterior surface. Lateral distances between the first exterior surface and the sidewall of the gate electrode are substantially the same. Lateral distances between the second exterior surface and the sidewall of the gate electrode increase along a direction from a top of the lower portion towards the semiconductor substrate.