A method of migrating a virtual machine having a virtual device that is backed by direct passthrough hardware, from a source host to a destination host, includes the steps of determining whether or not the destination host has direct passthrough hardware that can back the virtual device, and upon determining that the destination host has direct passthrough hardware that can back the virtual device, determining if a version of the direct passthrough hardware at the source host matches a version of the direct passthrough hardware at the destination host. If the versions do not match, the steps further include quiescing the virtual device, deleting data structures relating to the virtual device, and then migrating the virtual machine from the source host to the destination host. If the versions match, the virtual machine is migrated without quiescing the virtual device and without deleting the data structures relating to the virtual device.