A method for forming an image sensor device is provided. The method includes forming a photodetector in a semiconductor substrate and forming a shielding layer over the semiconductor substrate. The method also includes forming a dielectric layer over the shielding layer and partially removing the dielectric layer to form a recess. The method further includes partially removing the shielding layer through the recess. In addition, the method includes forming a filter in the recess after the shielding layer is partially removed.