Metadata of extent-based storage systems can be managed. For example, a computing device can store a first metadata object and a second metadata object in a first memory device. The first metadata object can specify locations of a first set of extents corresponding to a first data unit stored in a second memory device. The second metadata object can specify locations of a second set of extents corresponding to a second data unit stored in the second memory device. The computing device can determine that a first size of the first metadata object is smaller than a second size of the second metadata object. The computing device can remove the second metadata object from the first memory device based on determining that the first size is less than the second size.