A first semiconductor die includes a first substrate, first semiconductor devices, first dielectric material layers having a first silicon oxide surface as an uppermost surface and forming first metal interconnect structures. A second semiconductor die includes a second substrate, second semiconductor devices, and second dielectric material layers forming second metal interconnect structures. A handle substrate is attached to a topmost surface of the second semiconductor die. The second substrate is thinned, and a second silicon oxide surface is provided as a bottommost surface of the second semiconductor die. The second semiconductor die is bonded to the first semiconductor die by inducing oxide-to-oxide bonding between the second silicon oxide surface and the first silicon oxide surface. The handle substrate is detached, and inter-die connection via structures are formed through the second substrate and the bonding interface to contact the first metal interconnect structures. External bonding pads may be subsequently formed.