A semiconductor device includes a substrate having a front side and a back side opposite to each other. A plurality of photodetectors is disposed in the substrate within a pixel region. An isolation structure is disposed within the pixel region and between the photodetectors. The isolation structure includes a back side isolation structure extending from the back side of the substrate to a position in the substrate. A conductive plug structure is disposed in the substrate within a periphery region. A conductive cap is disposed on the back side of the substrate and extends from the pixel region to the periphery region and electrically connects the back side isolation structure to the conductive plug structure. A conductive contact lands on the conductive plug structure, and is electrically connected to the back side isolation structure through the conductive plug structure and the conductive cap.