Systems, apparatuses and methods may provide for a frontend driver that notifies a hypervisor of a map request from a guest driver of a device, wherein the device is passed through to and directly controlled by a virtual machine, and wherein the map request is associated with an attempt of the device to access a guest memory page in a virtualized execution environment. The frontend driver may also determine whether the guest memory page is pinned and send a map hypercall to the hypervisor if the guest memory page is not pinned. Additionally, the hypervisor may determine that the guest memory page is pinned, determine, based on a direct memory access (DMA) bitmap, that an unmap request from the guest driver has been issued, and unpin the guest memory page.