Techniques for dynamically using system (i.e., VM guest) memory as video memory for virtual graphics processing units (VGPUs) are provided. In one embodiment, a guest graphics driver running within a virtual machine (VM) of a host system can receive, from a guest application of the VM, a request to create a graphics resource. The guest graphics driver can then dynamically allocate, in response to the request, a memory object for the graphics resource in a guest memory space of the VM.