Techniques and structures relating to virtual graphics processing units (VGPUs) are disclosed. A VGPU may appear to software as an independent hardware GPU. However, two or more VGPUs can be implemented on the same GPU through the use of control structures and by duplicating some (but not all) hardware elements of the GPU. For example, additional registers and storage space may be added in a GPU supporting multiple VGPUs. Different execution priorities may be set for tasks and threads that correspond to the different supported VGPUs. Memory address space for the VGPUs may also be managed, including use of virtual address space for different VGPUs. Halting and resuming execution of different VGPUs allows for fine-grained execution control in various embodiments.