A semiconductor substrate 2 is dry etched before an insulating layer 4 is exposed, whereby a hole H1 penetrating through the semiconductor substrate 2 and reaching the insulating layer 4 is formed at a position corresponding to a photosensitive region S1. Next, an irregular asperity 22 is formed in a surface 7 of an n+ type embedded layer 6 exposed in the hole H1. The surface of the n+ type embedded layer 6 exposed in the hole H1 through the insulating layer 4 is irradiated with a picosecond to femtosecond pulsed laser beam, whereby the insulating layer 4 is removed and the surface 7 of the n+ type embedded layer 6 exposed in the hole H1 is roughened by the picosecond to femtosecond pulsed laser beam, to form the irregular asperity 22 in the entire area of the surface 7. Then the substrate with the irregular asperity 22 therein is subjected to a thermal treatment.