A three-dimensional memory device includes an alternating stack of insulating layers and electrically conductive layers located over a substrate, memory openings vertically extending through the alternating stack, and memory stack structures extending through the alternating stack. Each of the memory stack structures contains a memory film and a vertical semiconductor channel. At least one of the electrically conductive layers contains a first conductive material portion having a respective inner sidewall that contacts a respective one of the memory films at a vertical interface, and a second conductive material portion that has a different composition from the first conductive material portion, and contacting the first electrically conductive material portion. The first conductive material portion has a lower work function than the second conductive material portion.