Methods, systems, and computer-readable storage media for online defragmentation of memory in database systems by applying an IX-lock to each table having data stored in a marked page in a set of marked pages, generating a record map including key-value pairs, each being associated with a record location in a marked page, a value of each key-value pair initially set to a first value, iteratively executing the online defragmentation to delete data from marked pages and add the data to non-sparse pages, at least one iteration including applying a try-lock to a record in a marked page, and at iterations of the online defragmentation, updating the record map to change the value of at least one key-value pair from the first value to the second value, the second value representing that data of a marked page has been deleted from the marked page and added to a non-sparse page.