A fabrication process is provided for a 3D stacked non-volatile memory device which provides a source contact to a bottom of a memory hole in a stack without exposing a programmable material lining of an interior sidewall of the memory hole and without exposing a channel forming region also lining an interior of the memory hole to an energetic and potentially damaging etch environment. The stack includes alternating control gate layers and dielectric layers on a substrate, and the memory hole is etched through the stack before lining an interior sidewall thereof with the programmable material and then with the channel forming material. The process avoids a need to energetically etch down through the memory hole to open up a source contact hole near the bottom of the channel forming material by instead etching upwardly from beneath the memory hole.