A method for preventing non-temporal entries from entering small critical structures is disclosed. The method comprises transferring a first entry from a higher level memory structure to an intermediate buffer. It further comprises determining a second entry to be evicted from the intermediate buffer and a corresponding value associated with the second entry. Subsequently, responsive to a determination that the second entry is frequently accessed, the method comprises installing the second entry into a lower level memory structure. Finally, the method comprises installing the first entry into a slot previously occupied by the second entry in the intermediate buffer.