A semiconductor memory device has memory cells for storing data, sense amplifiers for amplifying the stored data, and cache cells in which the amplified data can be placed for quick recall. The cache cells can continue to hold data during memory-cell refresh cycles, permitting the cached data to be accessed quickly afterward. The cache cells may be coupled to column data lines that can be disconnected from the sense amplifiers, enabling memory cells to be refreshed while cache access is in progress. Write buffers may be provided so that when cache data are replaced, the old cache data can be copied back to the memory cells while the new cache data are being accessed.