A semiconductor device and a method of forming the same are provided. The semiconductor device includes a substrate, a gate structure, a dielectric structure and a contact structure. The substrate has source/drain (S/D) regions. The gate structure is on the substrate and between the S/D regions. The dielectric structure covers the gate structure. The contact structure penetrates through the dielectric structure to connect to the S/D region. A lower portion of a sidewall of the contact structure is spaced apart from the dielectric structure by an air gap therebetween, while an upper portion of the sidewall of the contact structure is in contact with the dielectric structure.