A semiconductor structure includes an alternating stack of insulating layers and electrically conductive layers located over a substrate, memory openings vertically extending through the alternating stack, memory opening fill structures located in the memory openings, where each of the memory opening fill structures contains a memory film and a vertical semiconductor channel that extend vertically, and each memory film includes a crystalline blocking dielectric metal oxide layer, and a metal oxide amorphous dielectric nucleation layer located between each of the vertically neighboring electrically conductive layers and insulating layers, and located between each of the crystalline blocking dielectric metal oxide layers and each of the electrically conductive layers.