公开(公告)号：US20210064581A1

公开(公告)日：2021-03-04

申请号：US16552976

申请日：2019-08-27

Applicant: VMware, Inc.

Inventor： Wenguang WANG ,  Junlong GAO ,  Marcos K. AGUILERA ,  Richard P. SPILLANE ,  Christos KARAMANOLIS ,  Maxime AUSTRUY

IPC: G06F16/174 ,  G06F16/13 ,  G06F16/172

Abstract: The present disclosure provides techniques for deduplicating files. The techniques include creating a cache or subset of a large data structure. The large data structure organizes information by random hash values. The random hash values result in a random organization of information within the data structure, with the information spanning a large number of storage blocks within a storage system. The cache, however, is within memory and is small relative to the data structure. The cache is created so as to contain information that is likely to be needed during deduplication of a file. Having needed information within memory rather than in storage results in faster read and write operations to that information, improving the performance of a computing system.

公开(公告)号：US20210064579A1

公开(公告)日：2021-03-04

申请号：US16552908

申请日：2019-08-27

Applicant: VMware, Inc.

Inventor： Wenguang WANG ,  Junlong GAO ,  Marcos K. AGUILERA ,  Richard P. SPILLANE ,  Christos KARAMANOLIS ,  Maxime AUSTRUY

IPC: G06F16/174 ,  G06F16/14

Abstract: Disclosed techniques include deduplication. Techniques include determining whether a file is unique, and depending on whether the file is unique, deduplicating only part of the file or the entire file. The techniques include processing the first chunk of a file to determine whether the hash of the chunk hash is already within a chunk hash table, and if not, then a percentage of chunks of the file is similarly processed. If any of the hashes of chunks are already in the chunk hash table, then at least some of file has been previously deduplicated, and file is not unique the storage system. If none of the processed chunks have a hash that is already in the chunk hash table, then the file is considered to be unique within chunk store and only a partial percentage of the file's chunks are deduplicated. Not all of a unique file's chunks are deduplicated.

公开(公告)号：US20210064522A1

公开(公告)日：2021-03-04

申请号：US16552954

申请日：2019-08-27

Applicant: VMware, Inc.

Inventor： Wenguang WANG ,  Junlong GAO ,  Marcos K. AGUILERA ,  Richard P. SPILLANE ,  Christos KARAMANOLIS ,  Maxime AUSTRUY

IPC: G06F12/02 ,  G06F3/06

Abstract: The present disclosure provides techniques for deallocating previously allocated storage blocks. The techniques include obtaining a list of chunk IDs to analyze, choosing a chunk ID, and determining the storage blocks spanned by the chunk corresponding to the chosen chunk ID. The technique further includes determining whether any file references any storage blocks spanned by the chunk. The determining may be performed by comparing an internal reference count to a total reference count, where the internal reference count is the number of reference to the storage block by a chunk ID data structure. If no files reference any of the storage blocks spanned by the chunk, then all the storage blocks of the chunk can be deallocated.

公开(公告)号：US20230409366A1

公开(公告)日：2023-12-21

申请号：US17840907

申请日：2022-06-15

Applicant: VMware, Inc.

Inventor： Jianqiao SUN ,  Yang YANG ,  Junlong GAO ,  Jianrong ZHAO ,  Jin FENG ,  Chengmao LU

IPC: G06F9/455

CPC classification number: G06F9/45558 ,  G06F2009/45595 ,  G06F2009/45591

Abstract: Example methods and systems associated with dynamic configuration of a statistics endpoint in a virtualized computing environment have been disclosed. One example method includes in response to receiving a first request, by a host in the virtualized computing environment, accepting a configuration file specified in the first request; in response to receiving a second request, by the host, parsing a rule based on the configuration file and collecting statistics based on the rule; processing, by the host, the statistics collected based on the rule; and sending, by the host, the processed statistics to a monitoring terminal.

公开(公告)号：US20220164125A1

公开(公告)日：2022-05-26

申请号：US17100663

申请日：2020-11-20

Applicant: VMware, Inc.

Inventor： Wenguang WANG ,  Petr VANDROVEC ,  Hardik Singh NEGI ,  Junlong GAO ,  Vamsi GUNTURU

IPC: G06F3/06

Abstract: A method for cleaning an object storage having a plurality of segments is provided. Each segment includes an identifier through which the segment is accessed. The method identifies a first segment in the plurality of segments. The first segment includes a first identifier and a first size. The method determines that a utilization ratio for the first segment is below a threshold. As a result, the method generates a second segment from the first segment, such that the second segment includes a second identifier that is the same as the first identifier and a second size that is smaller than the first size. The method then writes the second segment to the object storage.

公开(公告)号：US20220156231A1

公开(公告)日：2022-05-19

申请号：US17097473

申请日：2020-11-13

Applicant: VMware, Inc.

Inventor： Wenguang WANG ,  Junlong GAO ,  Vamsi GUNTURU

IPC: G06F16/18 ,  G06F16/188 ,  G06F16/182 ,  G06F16/16 ,  G06F16/901 ,  G06F9/455 ,  G06F11/14

Abstract: A method for managing data associated with objects stored in a cloud storage is provided. The method receives, at a first compute node, first data associated with an object stored in the cloud storage, the first compute node being one of a plurality of compute nodes that store data associated with different objects as storage objects in a log-structured merging (LSM) tree data structure. The method then assigns a first unique name to a first storage object associated with the first data, the first unique name comprising a combination of at least an identifier identifying the first compute node and a first incremental local value. The method stores the first storage object in a first level (L0) of the LSM tree data structure.

公开(公告)号：US20210064589A1

公开(公告)日：2021-03-04

申请号：US16552880

申请日：2019-08-27

Applicant: VMware, Inc.

Inventor： Wenguang WANG ,  Junlong GAO ,  Marcos K. AGUILERA ,  Richard P. SPILLANE ,  Christos KARAMANOLIS ,  Maxime AUSTRUY

IPC: G06F16/215 ,  G06F16/22

Abstract: The present disclosure provides techniques for scaling out deduplication of files among a plurality of nodes. The techniques include designating a master component for the coordination of deduplication. The master component divides files to be deduplicated among several slave nodes, and provides to each slave node a set of unique identifiers that are to be assigned to chunks during the deduplication process. The techniques herein preserve integrity of the deduplication process that has been scaled out among several nodes. The scaled out deduplication process deduplicates files faster by allowing several deduplication modules to work in parallel to deduplicate files.

公开(公告)号：US20200183905A1

公开(公告)日：2020-06-11

申请号：US16212550

申请日：2018-12-06

Applicant: VMware, Inc.

Inventor： Wenguang WANG ,  Richard P. SPILLANE ,  Junlong GAO ,  Robert T. JOHNSON ,  Christos KARAMANOLIS ,  Maxime AUSTRUY

IPC: G06F16/22 ,  H03M7/30

Abstract: Certain aspects provide systems and methods of compacting data within a log-structured merge tree (LSM tree) using sharding. In certain aspects, a method includes determining a size of the LSM tree, determining a compaction time for a compaction of the LSM tree based on the size, determining a number of compaction entities for performing the compaction in parallel based on the compaction time, determining a number of shards based on the number of compaction entities, and determining a key range associated with the LSM tree. The method further comprises dividing the key range by the number of shards into a number of sub key ranges, wherein each of the number of sub key ranges corresponds to a shard of the number of shards and assigning the number of shards to the number of compaction entities for compaction.