Some embodiments provide a non-transitory machine-readable medium that stores a program. The program receives a request for a logical data structure of an in-memory database. The program further determines a size of fragments of memory for the logical data structure based on a set of criteria. The program also requests a set of segments of memory. Upon receiving the set of segments of memory, the program further generates a plurality of fragments of memory from the set of segments of memory based on the size of fragments of memory. The program also groups the plurality of fragments of memory into the logical data structure. The plurality of fragments of memory are configured to store data of the logical data structure. The program further loads the plurality of fragments of memory into memory.