公开(公告)号：US11409705B2

公开(公告)日：2022-08-09

申请号：US16517436

申请日：2019-07-19

Applicant: VMware, Inc.

Inventor： William Earl ,  Christos Karamanolis ,  Kiran Joshi

IPC: G06F16/17 ,  G06F11/14

Abstract: Embodiments of the disclosure provide techniques managing a log-structured solid state drive (SSD) format in a distributed storage system. SSDs in the distributed storage system maintains a journal of logical changes to storage objects to persist prepared and committed changes in the latency path. The journal includes metadata entries that describe changes and reference data pages. Dense data structures (such as a logical block addressing table) index the metadata entries. To reduce the amount of overhead in I/O operations, the distributed storage system maintains the dense data structures in memory rather than on disk.

公开(公告)号：US11176099B2

公开(公告)日：2021-11-16

申请号：US16231300

申请日：2018-12-21

Applicant: VMware, Inc.

Inventor： Wenguang Wang ,  Junlong Gao ,  Richard P. Spillane ,  Robert T. Johnson ,  Christos Karamanolis ,  Maxime Austruy

IPC: G06F16/178 ,  G06F16/18

Abstract: The disclosure herein describes synchronizing a data cache and an LSM tree file system on an object storage platform. Instructions to send a cached data set from the data cache to the LSM tree file system are received. An updated metadata catalog is generated. If the LSM tree structure is out of shape, compaction is performed on the LSM tree file system which may be on a different system or server. When an unmerged compacted metadata catalog is identified, a merged metadata catalog is generated, based on the compacted metadata catalog and the cached data set, and associated with the cached data set. The cached data set and the associated metadata catalog are sent to the LSM tree file system, whereby the data cache and the LSM tree file system are synchronized. Synchronization is enabled without the data cache or file system being locked and/or waiting for the other entity.

公开(公告)号：US11074225B2

公开(公告)日：2021-07-27

申请号：US16231246

申请日：2018-12-21

Applicant: VMware, Inc.

Inventor： Wenguang Wang ,  Richard P. Spillane ,  Junlong Gao ,  Robert T. Johnson ,  Christos Karamanolis ,  Maxime Austruy

IPC: G06F7/00 ,  G06F16/178 ,  G06F16/13 ,  G06F16/172 ,  G06F16/185

Abstract: The disclosure herein describes synchronizing cached index copies at a first site with indexes of a log-structured merge (LSM) tree file system on an object storage platform at a second site. An indication that the LSM tree file system has been compacted based on a compaction process is received. A cached metadata catalog of the included parent catalog version at the first site is accessed. A set of cached index copies is identified at the first site based on the metadata of the cached metadata catalog. The compaction process is applied to the identified set of cached index copies and a compacted set of cached index copies is generated at the first site, whereby the compacted set of cached index copies is synchronized with a respective set of indexes of the plurality of sorted data tables of the LSM tree file system at the second site.

公开(公告)号：US20210075855A1

公开(公告)日：2021-03-11

申请号：US17062411

申请日：2020-10-02

Applicant: VMware, Inc.

Inventor： Richard Spillane ,  Yunshan Luke Lu ,  Wenguang Wang ,  Maxime Austruy ,  Christos Karamanolis ,  Rawlinson Rivera

IPC: H04L29/08 ,  G06F9/455 ,  G06F16/11 ,  G06F16/178

Abstract: Examples disclosed herein relate to propagating changes made on a file system volume of a primary cluster of nodes to the same file system volume also being managed by a secondary cluster of nodes. An application is executed on both clusters, and data changes on the primary cluster are mirrored to the secondary cluster using an exo-clone file. The exo-clone file includes the differences between two or more snapshots of the volume on the primary cluster, along with identifiers of the change blocks and (optionally) state information thereof. Just these changes, identifiers, and state information are packaged in the exo-clone file and then exported to the secondary cluster, which in turn makes the changes to its version of the volume. Exporting just the changes to the data blocks and the corresponding block identifiers drastically reduces the information needed to be exchanged and processed to keep the two volumes consistent.

公开(公告)号：US10909102B2

公开(公告)日：2021-02-02

申请号：US16212550

申请日：2018-12-06

Applicant: VMware, Inc.

Inventor： Wenguang Wang ,  Richard P. Spillane ,  Junlong Gao ,  Robert T. Johnson ,  Christos Karamanolis ,  Maxime Austruy

IPC: G06F16/00 ,  G06F16/22 ,  H03M7/30

Abstract: Certain aspects provide systems and methods of compacting data within a log-structured merge tree (LSM tree) using sharding. In certain aspects, a method includes determining a size of the LSM tree, determining a compaction time for a compaction of the LSM tree based on the size, determining a number of compaction entities for performing the compaction in parallel based on the compaction time, determining a number of shards based on the number of compaction entities, and determining a key range associated with the LSM tree. The method further comprises dividing the key range by the number of shards into a number of sub key ranges, wherein each of the number of sub key ranges corresponds to a shard of the number of shards and assigning the number of shards to the number of compaction entities for compaction.

公开(公告)号：US10776045B2

公开(公告)日：2020-09-15

申请号：US15666343

申请日：2017-08-01

Applicant: VMware, Inc.

Inventor： Richard P. Spillane ,  Wenguang Wang ,  Abhishek Gupta ,  Maxime Austruy ,  Christos Karamanolis

IPC: G06F3/06 ,  G06F12/02

Abstract: System and method for managing multiple data storages using a file system of a computer system utilize a primary data storage to cache objects of logical object containers stored in a secondary data storage in caching-tier volumes. When an access request for an object stored in the secondary data storage is received at the file system and the object is not currently cached in the primary data storage, a caching-tier volume in the primary data storage is created that corresponds to a logical object container in the secondary data storage that includes the requested object. The caching-tier volume is used to cache the object as an inflated file so that the inflated file is available at the primary data storage in the caching-tier volume for a subsequent access request for the object stored in the secondary data storage.

公开(公告)号：US10747475B2

公开(公告)日：2020-08-18

申请号：US14010316

申请日：2013-08-26

Applicant: VMware, Inc.

Inventor： Christos Karamanolis ,  Mansi Shah ,  Nathan Burnett

IPC: G06F3/06

Abstract: Techniques are described for storing a virtual disk in an object store comprising a plurality of physical storage devices housed in a plurality of host computers. A profile is received for creation of the virtual disk wherein the profile specifies storage properties desired for an intended use of the virtual disk. A virtual disk blueprint is generated based on the profile such that that the virtual disk blueprint describes a storage organization for the virtual disk that addresses redundancy or performance requirements corresponding to the profile. A set of the physical storage devices that can store components of the virtual disk in a manner that satisfies the storage organization is then determined.

公开(公告)号：US20200183906A1

公开(公告)日：2020-06-11

申请号：US16213714

申请日：2018-12-07

Applicant: VMware, Inc.

Inventor： Richard P. Spillane ,  Wenguang Wang ,  Junlong Gao ,  Robert T. Johnson ,  Christos Karamanolis ,  Maxime Austruy

IPC: G06F16/22 ,  G06F16/13 ,  G06F16/2455 ,  G06F16/84

Abstract: The disclosure herein describes providing and accessing data on an object storage platform using a log-structured merge (LSM) tree file system. The LSM tree file system on the object storage platform includes sorted data tables, each sorted data table including a payload portion and an index portion. Data is written to the LSM tree file system in at least one new sorted data table. Data is ready by identifying a data location of the data based on index portions of the sorted data tables and reading the data from a sorted data table associated with the identified data location. The use of the LSM tree file system on the object storage platform provides an efficient means for interacting with the data stored thereon.

公开(公告)号：US20200183886A1

公开(公告)日：2020-06-11

申请号：US16213815

申请日：2018-12-07

Applicant: VMware, Inc.

Inventor： Wenguang Wang ,  Richard P. Spillane ,  Junlong Gao ,  Robert T. Johnson ,  Christos Karamanolis ,  Maxime Austruy

IPC: G06F16/172 ,  G06F16/174 ,  G06F16/901 ,  G06F12/0891

Abstract: The disclosure herein describes writing data to a log-structured merge (LSM) tree file system on an object storage platform. Write data instructions indicating data for writing to the LSM tree file system are received. Based on the received instructions, the data is written to the first data cache. Based on an instruction to transfer data in the live data cache to the LSM tree file system, the first data cache is converted to a stable cache. A second data cache configured as a live data cache is then generated based on cloning the first data cache. The data in the first data cache is then written to the LSM tree file system. Use of a stable cache and a cloned live data cache enables parallel writing data to the file system by the stable cache and handling write data instructions by the live data cache.

公开(公告)号：US20190004735A1

公开(公告)日：2019-01-03

申请号：US15666343

申请日：2017-08-01

Applicant: VMware, Inc.

Inventor： Richard P. Spillane ,  Wenguang Wang ,  Abhishek Gupta ,  Maxime Austruy ,  Christos Karamanolis

IPC: G06F3/06

Abstract: System and method for managing multiple data storages using a file system of a computer system utilize a primary data storage to cache objects of logical object containers stored in a secondary data storage in caching-tier volumes. When an access request for an object stored in the secondary data storage is received at the file system and the object is not currently cached in the primary data storage, a caching-tier volume in the primary data storage is created that corresponds to a logical object container in the secondary data storage that includes the requested object. The caching-tier volume is used to cache the object as an inflated file so that the inflated file is available at the primary data storage in the caching-tier volume for a subsequent access request for the object stored in the secondary data storage.