There has been a problem in conventional Si-type floating-gate type nonvolatile semiconductor memory devices that the charge retention characteristic is low due to insufficiently large electron affinity of Si, therefore improvement of the memory performances, such as scaling down of a memory cell and increasing operation speed, have been difficult to be achieved due to the essential problem. In order to solve the above problem, in the nonvolatile semiconductor memory device of the present invention, a material having large work function or large electron affinity or a material having a work function close to that of semiconductor substrate or of a control gate, is employed for a floating gate retaining charges. Further, an amorphous material having small electron affinity for an insulating matrix is used. Further, at a time of deposition of charge retention layer, the supply ratio of the nano-particle material and the insulating matrix material, such as the mixture ratio of materials of both phases in a target in a sputtering method, is adjusted. By these methods, the charge retention characteristic of the floating-gate type nonvolatile semiconductor memory device can be improved, and the above-mentioned problem of the nonvolatile semiconductor memory device can be solved.