To provide a semiconductor memory device capable of fast rewriting with a small area, and/or large-capacity operation with a small area or fast operation and low power consumption operation, peripheral circuits such as logic circuit, buffer memory and sense circuit or part thereof are formed on a semiconductor substrate surface, and memory cells are provided thereon with an insulator film interposed therebetween.