Some aspects of the invention provide methods, systems, and computer program products for creating an in-memory physical dictionary for data compression. To that end, in accordance with aspects of the present invention, a new heuristic is defined for converting each of the plurality of logical nodes into a corresponding physical node forming a plurality of physical nodes; then place each of the physical nodes into the physical dictionary while traversing the dictionary tree in descending visit count order. Each physical node is placed in its nearest ascendant's cache-line with sufficient space. If there is no space in any of the ascendant's cache-line, then the physical node is placed into a new cache-line, unless a pre-defined packing threshold has been reached, in which case the physical node is placed in the first available cache-line.