An embedded flash memory device includes a gate stack, which includes a bottom dielectric layer extending into a recess in a semiconductor substrate, and a charge storage layer over the bottom dielectric layer. The charge storage layer includes a portion in the recess. The gate stack further includes a top dielectric layer over the charge storage layer, and a metal gate over the top dielectric layer. Source and drain regions are in the semiconductor substrate, and are on opposite sides of the gate stack.