An integrated circuit structure includes a semiconductor substrate; an interconnect structure over the semiconductor substrate, wherein the interconnect structure comprises a top inter-metal dielectric (IMD); an opening penetrating the interconnect structure into the semiconductor substrate; a conductor in the opening; and an isolation layer having a vertical portion and a horizontal portion physically connected to each other. The vertical portion is on sidewalls of the opening. The horizontal portion is directly over the interconnect structure. The integrated circuit structure is free from passivation layers vertically between the top IMD and the horizontal portion of the isolation layer.