A semiconductor substrate having an etch stop layer and at least a dielectric layer disposed from bottom to top is provided. The dielectric layer and the etching stop layer is then patterned to form a plurality of openings exposing the semiconductor substrate. A dielectric thin film is subsequently formed to cover the dielectric layer, the sidewalls of the openings, and the semiconductor substrate. The dielectric thin film disposed on the dielectric layer and the semiconductor substrate is then removed while the dielectric thin film disposed on the sidewalls remains.