There is disclosed a semiconductor memory device comprising memory cells (M11 to Mmn) for storing binary data, and first reference cells (DM11 to DMm1) and second reference cells (DM12 to DMm2) corresponding to respective two storage states of the memory cell, to make comparisons between the storage state of the memory cell and the storage states of the both reference cells at first and second sense amplifiers (1, 2) to compare outputs from the both sense amplifiers at the third sense amplifier (3) to thereby detect storage data of the memory cell. Thus, there can be provided a high speed memory device which has a less number of memory cells and of a high integration structure, and which has a little possibility of an erroneous operation in reading.