Reducing hash collisions in large scale data deduplication
摘要:
A system obtains a first data chunk and a second data chunk of a plurality of data chunks associated with a first data snapshot of a computing system. A hash record is assigned to a data chunk, and used to create a hash value that is written to a first lookup table. The hash function is selected from a plurality of hash functions. The system creates a first archive by saving the plurality of data chunks and the first lookup table to a datastore. The system writes a second hash record for the individual data chunks to a second lookup table using the same hash functions that were used for the first lookup table. Dissimilar hash values between the first lookup table and the second lookup table are identified, and a second archive that includes data chunks with different data from the corresponding data chunks from the first data snapshot is created based on the data chunks with the dissimilar hash values.
信息查询
0/0