A etch stop semiconductor rail is formed within a source semiconductor layer. A laterally alternating stack of dielectric rails and sacrificial semiconductor rails is formed over the source semiconductor layer and the etch stop semiconductor rail. After formation of a vertically alternating stack of insulating layers and spacer material layers, memory stack structures are formed through the vertically alternating stack and through interfaces between the sacrificial semiconductor rails and the dielectric rails. A backside trench is formed through the vertically alternating stack employing the etch stop semiconductor rail as an etch stop structure. Source strap rails providing lateral electrical contact to semiconductor channels of the memory stack structures are formed by replacement of sacrificial semiconductor rails with source strap rails.