The server based graphics processing techniques, describer herein, include loading a given instance of a guest shim layer and loading a given instance of a guest display device interface that calls back into the given instance of the guest shim layer, in response to loading the given instance of the guest shim layer, wherein the guest shim layer and the guest display device interface are executing under control of a virtual machine guest operating system. The given instance of the shim layer requests a communication channel between the given instance of the guest shim layer and a host-guest communication manager (D3D HGCM) service module from a host-guest communication manager (HGCM). In response to the request for the communication channel loading, the D3D HGCM service module is loaded and a communication channel between the given instance of the shim layer and the D3D HGCM service module is created by the HGCM. The given instance of the shim layer maps the graphics buffer memory space of a host D3D DDI binary executing under control of a host operating system. Thereafter, function calls are sent from the given instance of the guest shim layer through the communication channel to the D3D HGCM service module utilizing the graphics buffer memory space mapping.