A method includes forming an opening extending from a back surface of a semiconductor substrate to a metal pad on a front side of the semiconductor substrate, and forming a first conductive layer including a first portion overlapping active image sensors in the semiconductor substrate, a second portion overlapping black reference image sensors in the semiconductor substrate, and a third portion in the opening to contact the metal pad. A second conductive layer is formed over and contacting the first conductive layer. A first patterning step is performed to remove the first and the second portions of the second conductive layer, wherein the first conductive layer is used as an etch stop layer. A second patterning step is performed to remove a portion of the first portion of the first conductive layer. The second and the third portions of the first conductive layer remain after the second patterning step.