Embodiments of mechanisms for forming a memory device structure are provided. The memory device includes a first gate stack structure. The first gate stack structure includes a first dielectric layer over a semiconductor substrate. The first gate stack structure also includes a first floating gate over the first dielectric layer, and the first floating gate has a tip corner. The first gate stack structure further includes a second dielectric layer conformally covering an upper surface and sidewalls of the first floating gate. The second dielectric layer has a substantially uniform thickness. In addition, the first gate stack structure includes a first control gate over the second dielectric layer and partially over the first floating gate.