A semiconductor structure includes an alternating stack of insulating layers and electrically conductive layers located over a substrate, memory openings vertically extending through the alternating stack, memory opening fill structures located in the memory openings, where each of the memory opening fill structures includes a memory film and a vertical semiconductor channel that extend vertically, and each memory film includes a first metal oxide blocking dielectric layer, and a second metal oxide blocking dielectric layer located between each of the vertically neighboring electrically conductive layers and insulating layers, and located between each of the first metal oxide blocking dielectric layers and each of the electrically conductive layers.