A semiconductor structure includes an alternating stack of insulating layers and electrically conductive layers, a memory opening vertically extending through the alternating stack, and a memory opening fill structure located in the memory opening and containing a vertical semiconductor channel and a memory film. The memory film includes a tunneling dielectric layer, a continuous charge storage material layer vertically extending through a plurality of the electrically conductive layers, a vertical stack of discrete charge storage elements located at levels of the electrically conductive layers and contacting a respective surface segment of an outer sidewall of the continuous charge storage material layer, and a vertical stack of discrete blocking dielectric material portions containing silicon atoms and oxygen atoms and located at the levels of the electrically conductive layers and vertically spaced apart from each other.