Once or more embodiments manage access to data by accelerator systems in an out-of-core processing environment. In one embodiment, a request from an accelerator system is received for access to a given data set. An access context associated with the given data set is determined The accelerator system is dynamically configured, based on the access context that has been determined, based on the access context that has been determined, to one of access the given data set directly from the server system; locally store a portion of the given data set in a memory; and locally store all of the given data set in the memory.