A shallow trench is formed to extend into a handle substrate of a semiconductor-on-insulator (SOI) layer. A dielectric liner stack of a dielectric metal oxide layer and a silicon nitride layer is formed in the shallow trench, followed by deposition of a shallow trench isolation fill portion. The dielectric liner stack is removed from above a top surface of a top semiconductor portion, followed by removal of a silicon nitride pad layer and an upper vertical portion of the dielectric metal oxide layer. A divot laterally surrounding a stack of a top semiconductor portion and a buried insulator portion is filled with a silicon nitride portion. Gate structures and source/drain structures are subsequently formed. The silicon nitride portion or the dielectric metal oxide layer functions as a stopping layer during formation of source/drain contact via holes, thereby preventing electrical shorts between source/drain contact via structures and the handle substrate.