A semiconductor memory device includes a cell array unit having a plurality of banks each having a plurality of blocks, and a refresh controller configured to set at least one of the blocks as a test block, perform a refresh operation on the blocks except for the test block in a self-refresh operation period, determine a refresh period of the test block, and then set another one of the blocks as the test block.