A semiconductor memory device having a cross point structure includes a plurality of upper electrodes arranged to extend in one direction, and a plurality of lower electrodes arranged to extend in another direction at a right angle to the one direction of the upper electrodes. Memory materials are provided between the upper electrodes and the lower electrodes for storage of data. The memory materials are made of a perovskite material and arranged at the lower electrodes side of the corresponding upper electrode extending along the corresponding upper electrode.