Throttling resynchronization operations in a data store cluster based on I/O bandwidth limits

    公开(公告)号:US10860254B2

    公开(公告)日:2020-12-08

    申请号:US16387433

    申请日:2019-04-17

    Applicant: VMware, Inc.

    Abstract: The disclosure herein describes throttling resynchronization operations in a cluster of distributed data stores based on I/O bandwidth limits. Resynchronization operations are added to a queue as they are received. A cluster resource manager accesses a resynchronization operation in the queue. A transient capacity value is determined and an available capacity value is obtained. A resynchronization limit of the cluster is identified, wherein the resynchronization limit indicates a maximum percentage of I/O bandwidth of the cluster that is for use by resynchronization operations. Based on a sum of the resynchronization capacity value and the transient capacity value being less than or equal to a product of the resynchronization limit and the available capacity value, the cluster resource manager initiates the accessed resynchronization operation on the cluster, whereby free space is maintained in the cluster for use by user operations.

    Maintaining quorum of storage objects in nested levels of a distributed storage system

    公开(公告)号:US10346063B2

    公开(公告)日:2019-07-09

    申请号:US15356413

    申请日:2016-11-18

    Applicant: VMware, Inc.

    Abstract: Exemplary methods, apparatuses, and systems determine that quorum can be maintained for a storage object in a distributed storage system in the event a defined maximum number of first partitions in a first level of storage and a second partitions in a second level of storage fail. When it is determined that there are insufficient numbers of first partitions and/or second partitions, additional first partitions and/or second partitions are associated with the storage object in the distributed storage system. A number of votes is calculated for distribution and an allocation is defined for assigning the votes to each component and witness component of the storage object.

    Combining the metadata and data address spaces of a distributed storage object via a composite object configuration tree

    公开(公告)号:US11474719B1

    公开(公告)日:2022-10-18

    申请号:US17320023

    申请日:2021-05-13

    Applicant: VMware, Inc.

    Abstract: Techniques for combining the metadata and data address spaces of a distributed storage object are provided. In one set of embodiments, a distributed storage system can receive a request to provision a storage object. In response, the distributed storage system can create, in accordance with an erasure coding scheme, one or more capacity components for holding data of the storage object; create, in accordance with a mirroring scheme having an equivalent level of fault tolerance as the erasure coding scheme, one or more metadata components for holding metadata of the storage object; and create a composite object configuration tree for the storage object that includes first and second subtrees, where the first subtree comprises an indication of the mirroring scheme and references to the one or more metadata components, and where the second subtree comprises an indication of the erasure coding scheme and references to the one or more capacity components.

    Identifying fault domains for delta components of a distributed data object

    公开(公告)号:US11422904B2

    公开(公告)日:2022-08-23

    申请号:US17106004

    申请日:2020-11-27

    Applicant: VMware, Inc.

    Abstract: The disclosure herein describes placing delta components of a base component in target fault domains. One or more delta components are generated. When a first fault domain that lacks a sibling component of the base component is identified, the first fault domain is selected as a single delta target fault domain and a single delta component is placed on the single delta target fault domain. When a second fault domain that includes a first sibling component of the base component is identified and a third fault domain that includes a second sibling component of the base component is identified, the second fault domain and the third fault domain are selected as a first double delta target fault domain and a second double delta target fault domain, and a first double delta component and a second double delta component are placed on the first and second double delta target fault domains.

    Reconfiguring data objects using batch processing of concatenated components

    公开(公告)号:US10949089B2

    公开(公告)日:2021-03-16

    申请号:US16442889

    申请日:2019-06-17

    Applicant: VMware, Inc.

    Abstract: The disclosure herein describes reconfiguring a data object in a data store using batch portions of components of the data object. Instructions to reconfigure the data object to a target object layout are received, the data object including a concatenation layer that references a plurality of components. The components are grouped into batch portions and the batch portions are reconfigured in a series based on the received instructions. The reconfiguring for each batch portion includes allocating free storage space according to the target object layout as target components, copying the data stored in the batch portion to the target components, redirecting references in the concatenation layer from the components of the batch portion to the target components, and releasing storage space occupied by the components of the batch portion. As a result, storage space used as transient capacity during the reconfiguring is reduced to a size of the batch portions.

    THROTTLING RESYNCHRONIZATION OPERATIONS IN A DATA STORE CLUSTER BASED ON I/O BANDWIDTH LIMITS

    公开(公告)号:US20200333977A1

    公开(公告)日:2020-10-22

    申请号:US16387433

    申请日:2019-04-17

    Applicant: VMware, Inc.

    Abstract: The disclosure herein describes throttling resynchronization operations in a cluster of distributed data stores based on I/O bandwidth limits. Resynchronization operations are added to a queue as they are received. A cluster resource manager accesses a resynchronization operation in the queue. A transient capacity value is determined and an available capacity value is obtained. A resynchronization limit of the cluster is identified, wherein the resynchronization limit indicates a maximum percentage of I/O bandwidth of the cluster that is for use by resynchronization operations. Based on a sum of the resynchronization capacity value and the transient capacity value being less than or equal to a product of the resynchronization limit and the available capacity value, the cluster resource manager initiates the accessed resynchronization operation on the cluster, whereby free space is maintained in the cluster for use by user operations.

    Dynamic object policy reconfiguration mechanism for object storage system

    公开(公告)号:US11693559B2

    公开(公告)日:2023-07-04

    申请号:US17212125

    申请日:2021-03-25

    Applicant: VMware, Inc.

    CPC classification number: G06F3/0604 G06F3/067 G06F3/0629

    Abstract: A method for dynamic storage object configuration in a datacenter is provided. Embodiments include determining a number of fault domains in a storage cluster that have sufficient storage capacity for creating a storage object. Embodiments include applying a dynamic fault tolerance policy to the number of fault domains that have sufficient capacity for creating the storage object in order to determine a number of host failures to tolerate for the storage object, the dynamic fault tolerance policy specifying a manner of determining, for any respective storage object, a respective number of host failures to tolerate for storing the respective storage object in a respective storage cluster based on at least a respective number of fault domains of the respective storage cluster. Embodiments include implementing the storage object on the storage cluster based on the number of host failures to tolerate for the storage object.

    Synchronizing a stale component of a distributed object using multiple delta components during maintenance

    公开(公告)号:US11182261B1

    公开(公告)日:2021-11-23

    申请号:US17106050

    申请日:2020-11-27

    Applicant: VMware, Inc.

    Abstract: The disclosure herein describes enhancing data durability of a base component using multiple delta components. A first and second delta component are generated based on the base component becoming unavailable. A write operation targeted for the base component is routed to the first delta component and to the second delta component. Based on routing the write operation targeted for the base component to the first delta component and to the second delta component, a bit associated with a data block affected by the write operation is changed in each of the tracking bitmaps of the first and second delta components. Based on detecting the base component becoming available, one delta component of the first and second delta components is selected, and the data block affected by the routed write operation is synchronized from the selected delta component to the base component. Later, the first and second delta components are removed.

Patent Agency Ranking