A mechanism is described for managing the caches on nodes in a cluster. The caches are globally managed so that a data item may be retained in any cache on the nodes. This may be accomplished by, for example, a replacement policy for replacing data items stored in the buffers of the caches, where a buffer is selected for replacement in a manner that accounts for factors that include the state of the caches of other nodes. Some cached data items are designated as globally shared, and assigned (either statically or dynamically) a primary cache. For example, if a buffer holds a copy of a data item whose primary cache is another node, then the data item in the buffer is favored for replacement over a local data item or a global data item for which this is the primary cache. According to another aspect of the invention, the cache retention value of buffers on different nodes are compared, and the buffer with the lowest cache retention value is selected for replacement. According to yet another aspect of the present invention, the replacement policy accounts for the configuration of other caches in the cluster.