A method of fabricating a semiconductor device includes the steps of: forming at least one first semi­conductor region (51) of a first conductivity type and at least one second semiconductor region (50) of a second conductivity type in a main surface of a semiconductor layer of the first conductivity type; forming a three-layer film having a desired shape on each of the first and second semiconductor regions, the three-layer film being made up of a bottom layer which is a conductive film (56), an intermediate layer which is a silicon nitride film (57), and a top layer which is a polycrystalline silicon film (58) doped with one of arsenic and phosphorus; forming a first insulating layer (61) on the side wall of the three-layer film; forming a second polycrystalline silicon film (62) on the whole surface, and diffusing one of arsenic and phosphorus from the first polycrystal­line silicon film (58) into the second polycrystalline silicon film (62); selectively etching off the first polycrystalline silicon film (58) and that portion (62A) of the second polycrystalline silicon film, in which one of arsenic and phosphorus has been diffused; forming a second insulating layer (64) at least on the surface of that portion (63A) of the second polycrystalline silicon film which exists on the second semiconductor region (50); removing the silicon nitride film (57) and the conductive film (56) which exist on the second semiconductor region (50), while using the second insulating layer (64) as a mask, to form an aperture and forming a third polycrystalline silicon film (69) so that the aperture (68) is covered by the third polycrystalline silicon film.