In a method of manufacturing a stacked-type semiconductor device, firstly, a first semiconductor substrate having a first device formed thereon is covered with an interlayer insulating layer and a planarized polycrystalline silicon layer is formed on the interlayer insulating layer. The first semiconductor substrate and a second semiconductor substrate are joined together by putting the surface of the polycrystalline silicon layer in close contact with the surface of a refractory metal layer formed on the second semiconductor substrate, applying thermal treatment at 700.degree. C. or below and changing the refractory metal layer to silicide.