A plurality of subgroups with a least recently used (LRU) list of data elements associated with count variables. The LRU lists have a top entry to store a most recently used data element and a bottom entry to store a least recently used data element. If a data element is accessed, then increase the value of the count variable and move the accessed data element to the top entry of the LRU list of the subgroup associated with the data element. If the value of the count variable of the accessed data element of the top entry is greater than a value of a count variable of a data element of a bottom entry of a LRU list of a subgroup with a higher priority, then swap the data element of the bottom entry with the accessed data element of the top entry.