According to one embodiment, a semiconductor memory device includes the following configuration. First Lower word line drivers are arranged between adjacent mats, and first upper word line drivers are arranged between the first Lower word line drivers. Second Lower word line drivers are arranged between another adjacent mats, and second upper word line drivers are arranged between the second lower word line drivers. The first and second upper word line drivers are shared by the adjacent mats respectively.