A method of forming a three-dimensional memory device is provided. Insulating layers and sacrificial layers are stacked alternatively on a substrate. At least one first opening is formed through the insulating layers and the sacrificial layers. Protection layers are formed on surfaces of the sacrificial layers exposed by the sidewall of the first opening. A charge storage layer is formed on the sidewall of the first opening and covers the protection layers. A channel layer is formed on the charge storage layer. The sacrificial layers and the protection layers are replaced with electrode layers. A three-dimensional memory device is further provided.