A plurality of data records of a data set can be stored in a plurality of main part fragments, at least one of which is an old fragment stored on-disk. A number of one or more data records in the old fragment that have been marked for deletion can be determined to be greater than a threshold number, and the old fragment can be loaded into main system memory. A merge of the old fragment can be performed to remove the one or more data records marked for deletion.