A method of manufacturing a semiconductor memory device having stacked capacitors is disclosed. After forming a capacitor isolating layer on an insulation layer and forming a contact hole in the insulation layer, a first conductive layer is formed on the insulating layer and the capacitor isolating layer and on an inner surface of the contact hole. The first conductive layer is partially etched and removed by using an etch-back technique to be isolated into a first capacitor portion and a second capacitor portion. A dielectric layer is formed on the first conductive layer. A second conductive layer is formed on the dielectric layer.