A semiconductor memory device with a variable resistance element includes a plurality of active areas isolated from one another by an isolation layer formed in a substrate, a plurality of word lines crossing over the plurality of active areas, an auxiliary source line disposed between two selected word lines and commonly connected to at least two active areas among the plurality of active areas between the two selected word lines, and a plurality of contact plugs each connected to a corresponding active area.