A method of forming a semiconductor memory device, the semiconductor memory device includes a plurality of active areas, a shallow trench isolation, a plurality of trenches and a plurality of gates. The active areas are defined on a semiconductor substrate, and surrounded by the shallow trench isolation. The trenches are disposed in the semiconductor substrate, penetrating through the active areas and the shallow trench isolation, wherein each of the trenches includes a bottom surface and a saddle portion protruded therefrom in each active areas. The gates are disposed in the trenches respectively.