Embodiments of mechanisms for forming an image sensor device are provided. The image sensor device includes a semiconductor substrate and an isolation structure in the semiconductor substrate. The image sensor device also includes an active region in the semiconductor substrate and surrounded by the isolation structure. The active region includes a light sensing region and a doped region, and the doped region has a horizontal length and a vertical length. A ratio of the horizontal length to the vertical length is in a range from about 1 to about 4.