Structures and formation methods of a semiconductor device structure are provided. The semiconductor device structure includes a semiconductor substrate and a first gate stack and a second gate stack over the semiconductor substrate. The semiconductor device structure also includes a first doped structure over the semiconductor substrate and adjacent to the first gate stack. The first doped structure includes a III-V compound semiconductor material and a dopant. The semiconductor device structure further includes a second doped structure over the semiconductor substrate and adjacent to the second gate stack. The second doped structure includes the III-V compound semiconductor material and the dopant. One of the first doped structure and the second doped structure is an n-type semiconductor structure, and the other one of the first doped structure and the second doped structure is a p-type semiconductor structure.