To provide a semiconductor device having a photoelectric conversion element having a high sensitivity, causing less blooming, and capable of providing a highly reliable image. The semiconductor device has a semiconductor substrate, a first p type epitaxial layer, a second p type epitaxial layer, and a first photoelectric conversion element. The first p type epitaxial layer is formed over the main surface of the semiconductor substrate. The second p type epitaxial layer is formed so as to cover the upper surface of the first p type epitaxial layer. The first photoelectric conversion element is formed in the second p type epitaxial layer. The first and second p type epitaxial layers are each made of silicon and the first p type epitaxial layer has a p type impurity concentration higher than that of the second p type epitaxial layer.