A method, hybrid server system, and computer program product, for managing access to data stored on the hybrid server system. A memory system residing at a server is partitioned into a first set of memory managed by the server and a second set of memory managed by a set of accelerator systems. The set of accelerator systems are communicatively coupled to the server. The memory system comprises heterogeneous memory types. A data set stored within at least one of the first set of memory and the second set of memory that is associated with at least one accelerator system in the set of accelerator systems is identified. The data set is transformed from a first format to a second format, wherein the second format is a format required by the at least one accelerator system.