公开(公告)号：US11669495B2

公开(公告)日：2023-06-06

申请号：US16552908

申请日：2019-08-27

申请人： VMware, Inc.

发明人： Wenguang Wang ,  Junlong Gao ,  Marcos K. Aguilera ,  Richard P. Spillane ,  Christos Karamanolis ,  Maxime Austruy

IPC分类号： G06F7/00 ,  G06F16/174 ,  G06F16/14

CPC分类号： G06F16/1752 ,  G06F16/152

摘要： Disclosed techniques include deduplication. Techniques include determining whether a file is unique, and depending on whether the file is unique, deduplicating only part of the file or the entire file. The techniques include processing the first chunk of a file to determine whether the hash of the chunk hash is already within a chunk hash table, and if not, then a percentage of chunks of the file is similarly processed. If any of the hashes of chunks are already in the chunk hash table, then at least some of file has been previously deduplicated, and file is not unique the storage system. If none of the processed chunks have a hash that is already in the chunk hash table, then the file is considered to be unique within chunk store and only a partial percentage of the file's chunks are deduplicated. Not all of a unique file's chunks are deduplicated.

公开(公告)号：US11775484B2

公开(公告)日：2023-10-03

申请号：US16552965

申请日：2019-08-27

申请人： VMware, Inc.

发明人： Junlong Gao ,  Wenguang Wang ,  Marcos K. Aguilera ,  Richard P. Spillane ,  Christos Karamanolis ,  Maxime Austruy

IPC分类号： G06F16/174 ,  G06F16/901 ,  G06F16/14

CPC分类号： G06F16/1752 ,  G06F16/152 ,  G06F16/9027

摘要： The disclosure provides techniques for deduplicating files. The techniques include, upon creating or modifying a file, placing a logical timestamp of the current logical time, within a queue associated with the directory of the file. The techniques further include placing the logical timestamp within a queue of each parent directory of the directory of the file. To determine a set of files for deduplication, the techniques disclosed herein identify files that have been modified within a logical time range. The set of files modified within a logical time is identified by traversing directories of a storage system, the directories being organized within a tree structure. If a directory's queue does not contain a timestamp that is within the logical time range, then all child directories can be skipped over for further processing, such that no files within the child directories end up being within the set of files for deduplication.

公开(公告)号：US11698760B2

公开(公告)日：2023-07-11

申请号：US17481418

申请日：2021-09-22

申请人： VMWARE, INC.

发明人： Marcos K. Aguilera ,  Keerthi Kumar ,  Pramod Kumar ,  Pratap Subrahmanyam ,  Sairam Veeraswamy ,  Rajesh Venkatasubramanian

IPC分类号： G06F3/06

CPC分类号： G06F3/067 ,  G06F3/065 ,  G06F3/0611 ,  G06F3/0613 ,  G06F3/0619

摘要： Disclosed are various embodiments for improving the resiliency and performance of cluster memory. First, a computing device can submit a write request to a byte-addressable chunk of memory stored by a memory host, wherein the byte-addressable chunk of memory is read-only. Then, the computing device can determine that a page-fault occurred in response to the write request. Next, the computing device can copy a page associated with the write request from the byte-addressable chunk of memory to the memory of the computing device. Subsequently, the computing device can free the page from the memory host. Then, the computing device can update a page table entry for the page to refer to a location of the page in the memory of the computing device.

公开(公告)号：US20230168965A1

公开(公告)日：2023-06-01

申请号：US18101536

申请日：2023-01-25

申请人： VMware, Inc.

发明人： Marcos K. Aguilera ,  Keerthi Kumar ,  Pramod Kumar ,  Pratap Subrahmanyam ,  Sairam Veeraswamy ,  Rajesh Venkatasubramanian

IPC分类号： G06F11/10 ,  G06F11/07 ,  G06F3/06

CPC分类号： G06F11/1068 ,  G06F11/0772 ,  G06F3/0673 ,  G06F3/0659 ,  G06F3/0619

摘要： Disclosed are various embodiments for improving the resiliency and performance of clustered memory. A computing device can generate at least one parity page from at least a first local page and a second local page. The computing device can then submit a first write request for the first local page to a first one of a plurality of memory hosts. The computing device can also submit a second write request for the second local page to a second one of the plurality of memory hosts. Additionally, the computing device can submit a third write request for the parity page to a third one of the plurality of memory hosts.

公开(公告)号：US11461229B2

公开(公告)日：2022-10-04

申请号：US16552954

申请日：2019-08-27

申请人： VMware, Inc.

发明人： Wenguang Wang ,  Junlong Gao ,  Marcos K. Aguilera ,  Richard P. Spillane ,  Christos Karamanolis ,  Maxime Austruy

IPC分类号： G06F3/06 ,  G06F12/02

摘要： The present disclosure provides techniques for deallocating previously allocated storage blocks. The techniques include obtaining a list of chunk IDs to analyze, choosing a chunk ID, and determining the storage blocks spanned by the chunk corresponding to the chosen chunk ID. The technique further includes determining whether any file references any storage blocks spanned by the chunk. The determining may be performed by comparing an internal reference count to a total reference count, where the internal reference count is the number of reference to the storage block by a chunk ID data structure. If no files reference any of the storage blocks spanned by the chunk, then all the storage blocks of the chunk can be deallocated.

公开(公告)号：US11372813B2

公开(公告)日：2022-06-28

申请号：US16552998

申请日：2019-08-27

申请人： VMware, Inc.

发明人： Wenguang Wang ,  Junlong Gao ,  Marcos K. Aguilera ,  Richard P. Spillane ,  Christos Karamanolis ,  Maxime Austruy

IPC分类号： G06F16/17 ,  G06F16/174 ,  G06F16/13 ,  G06F16/172 ,  G06F9/455

摘要： The present disclosure provides techniques for deduplicating files. The techniques include creating a data structure that organizes metadata about chunks of files, the organization of the metadata preserving order and locality of the chunks within files. The organization of the metadata within storage blocks of storage devices matches the order of chunks within files. Upon a read or write operation to a metadata, the preservation of locality of metadata results in the likely fetching, from storage into a memory cache, metadata of subsequent and contiguous chunks. The preserved locality results in faster subsequent read and write operations of metadata, because the read and write operations are likely to be executed from memory rather than from storage.

公开(公告)号：US11055265B2

公开(公告)日：2021-07-06

申请号：US16552880

申请日：2019-08-27

申请人： VMware, Inc.

发明人： Wenguang Wang ,  Junlong Gao ,  Marcos K. Aguilera ,  Richard P. Spillane ,  Christos Karamanolis ,  Maxime Austruy

IPC分类号： G06F16/00 ,  G06F16/215 ,  G06F16/22

摘要： The present disclosure provides techniques for scaling out deduplication of files among a plurality of nodes. The techniques include designating a master component for the coordination of deduplication. The master component divides files to be deduplicated among several slave nodes, and provides to each slave node a set of unique identifiers that are to be assigned to chunks during the deduplication process. The techniques herein preserve integrity of the deduplication process that has been scaled out among several nodes. The scaled out deduplication process deduplicates files faster by allowing several deduplication modules to work in parallel to deduplicate files.

公开(公告)号：US12045204B2

公开(公告)日：2024-07-23

申请号：US16552976

申请日：2019-08-27

申请人： VMware, Inc.

发明人： Wenguang Wang ,  Junlong Gao ,  Marcos K. Aguilera ,  Richard P. Spillane ,  Christos Karamanolis ,  Maxime Austruy

IPC分类号： G06F15/16 ,  G06F16/13 ,  G06F16/172 ,  G06F16/174 ,  G06F9/455

CPC分类号： G06F16/1752 ,  G06F16/137 ,  G06F16/172 ,  G06F9/45558 ,  G06F2009/45583

摘要： The present disclosure provides techniques for deduplicating files. The techniques include creating a cache or subset of a large data structure. The large data structure organizes information by random hash values. The random hash values result in a random organization of information within the data structure, with the information spanning a large number of storage blocks within a storage system. The cache, however, is within memory and is small relative to the data structure. The cache is created so as to contain information that is likely to be needed during deduplication of a file. Having needed information within memory rather than in storage results in faster read and write operations to that information, improving the performance of a computing system.

公开(公告)号：US11704030B2

公开(公告)日：2023-07-18

申请号：US17481352

申请日：2021-09-22

申请人： VMWARE, INC.

发明人： Marcos K. Aguilera ,  Keerthi Kumar ,  Pramod Kumar ,  Pratap Subrahmanyam ,  Sairam Veeraswamy ,  Rajesh Venkatasubramanian

IPC分类号： G06F3/06

CPC分类号： G06F3/0631 ,  G06F3/0604 ,  G06F3/067 ,  G06F3/0659

摘要： Disclosed are various embodiments for improving resiliency and performance of clustered memory. A computing device can acquire a chunk of byte-addressable memory from a cluster memory host. The computing device can then identify an active set of allocated memory pages and an inactive set of allocated memory pages for a process executing on the computing device. Next, the computing device can store the active set of allocated memory pages for the process in the memory of the computing device. Finally, the computing device can store the inactive set of allocated memory pages for the process in the chunk of byte-addressable memory of the cluster memory host.

公开(公告)号：US11687286B2

公开(公告)日：2023-06-27

申请号：US17481335

申请日：2021-09-22

申请人： VMWARE, INC.

发明人： Marcos K. Aguilera ,  Keerthi Kumar ,  Pramod Kumar ,  Pratap Subrahmanyam ,  Sairam Veeraswamy ,  Rajesh Venkatasubramanian

IPC分类号： G06F12/00 ,  G06F12/02 ,  G06F3/06

CPC分类号： G06F3/0659 ,  G06F3/061 ,  G06F3/0604 ,  G06F3/067 ,  G06F3/0631

摘要： Disclosed are various embodiments for improving the resiliency and performance for clustered memory. A computing device can mark a page of the memory as being reclaimed. The computing device can then set the page of the memory as read-only. Next, the computing device can submit a write request for the contents of the page to individual ones of a plurality of memory hosts. Subsequently, the computing device can receive individual confirmations of a successful write of the page from the individual ones of the plurality of memory hosts. Then, the computing device can mark the page as free in response to receipt of the individual confirmations of the successful write from the individual ones of the plurality of memory hosts.