A method of manufacturing a semiconductor memory device having non-volatile memory elements or memory cells of a single-element type. The method provides for the formation of a floating gate electrode on a main surface of a semiconductor substrate and a control gate electrode on the floating gate electrode via a second gate insulating film. In accordance with the method, an impurity is introduced in self-alignment with one of a pair of opposing end portions of the control gate electrode to form a first semiconductor region, and on the second of the opposing end portions of the control gate electrode of the memory cell, the same impurity, for example, arsenic, but, however, of a lower dose is introduced in self-alignment to form a second semiconductor region. In accordance with the formation of the first semiconductor region, the impurity is selectively introduced into the substrate by using a mask layer which covers a portion of the main surface of the substrate where the second semiconductor region is to be formed. In accordance with such manufactured memory cells, carriers which are stored in the floating gate electrode are transferred therefrom to the first semiconductor region by tunneling through the insulating film underlying the floating gate electrode. The first semiconductor region is formed so as to extend in an overlapping relation with the floating gate electrode by a greater amount than that of the second semiconductor region.