Techniques for optimizing virtual machine (VM) storage performance in a hyper-converged infrastructure (HCI) deployment comprising a stretched cluster of host systems are provided. In one set of embodiments, a computer system can identify one or more stretched VMs in the stretched cluster, where storage objects associated with the one or more stretched VMs are replicated across the sites of the cluster. The computer system can further determine, for each stretched VM, whether a greater number of the VM's storage objects are accessible to the VM via site-local replica copies residing at a first site where the VM is currently running, or via site-remote replica copies residing at a second site where the VM is not currently running. If a greater number of the VM's storage objects are accessible to the VM via the site-remote replica copies, the VM can be migrated from the first site to the second site.