Sacrificial semiconductor material portions are connected by a sacrificial semiconductor line extending along a different horizontal direction and protruding into an underlying source conductive layer. After formation of a vertically alternating stack of insulating layers and spacer material layers, memory stack structures are formed through the vertically alternating stack and through the sacrificial semiconductor material portions. A backside trench can be formed through the vertically alternating stack employing the sacrificial semiconductor line as an etch stop structure. Source strap material portions providing lateral electrical contact to semiconductor channels of the memory stack structures can be formed by replacement of sacrificial semiconductor material portions and the sacrificial semiconductor line with source strap material portions. Structural-reinforcement portions may be employed to provide structural stability during the replacement process.