A semiconductor photodiode device includes a semiconductor substrate, a first buffer layer containing a material different from that of the semiconductor substrate in a portion thereof, a first semiconductor layer formed above the buffer layer and having a lattice constant different from that of the semiconductor substrate, a second buffer layer formed above the first semiconductor layer and containing an element identical with that of the first semiconductor layer in a portion thereof, and a second semiconductor layer formed above the buffer layer in which a portion of the first semiconductor layer is formed of a plurality of island shape portions each surrounded with an insulating film, and the second buffer layer allows adjacent islands of the first semiconductor layer to coalesce with each other and is in contact with the insulating film.