Flexible efficient runtime placement of data across multiple disks

    公开(公告)号:US10268408B2

    公开(公告)日:2019-04-23

    申请号:US15853110

    申请日:2017-12-22

    Applicant: VMware, Inc.

    Abstract: In a storage cluster having nodes, blocks of a logical storage space of a storage object are allocated flexibly by a parent node to component nodes that are backed by physical storage. The method includes maintaining a first allocation map for the parent node, and second and third allocation maps for the first and second component nodes, respectively, executing a first write operation on the first component node and updating the second allocation map to indicate that the first block is a written block, selecting the second component node for executing a second write operation, and executing the second write operation on the second component node. Upon execution of the second write operation, the third allocation map is updated to indicate that the second block is a written block and the first allocation map is updated to indicate that the second block is allocated to the second component node.

    SYNCHRONIZING REPLICAS WITH MEDIA ERRORS IN DISTRIBUTED STORAGE SYSTEMS

    公开(公告)号:US20180225171A1

    公开(公告)日:2018-08-09

    申请号:US15943293

    申请日:2018-04-02

    Applicant: VMware, Inc.

    CPC classification number: G06F11/1004 G06F3/0619 G06F3/064 G06F3/0683

    Abstract: Systems and techniques are described for transferring data. A described technique includes receiving a request to transmit a data block from a first data storage device to a second data storage device. An attempt to read the data block from the first data storage device is made. A media error resulting from the attempt to read the data block from the first data storage device is detected. In response to detecting the media error, a new data block is generated and includes mismatched checksum data that causes a checksum mismatched error when the new data block is accessed. The new data block is transmitted for storage at the second data storage device in place of the data block.

    Synchronizing replicas with media errors in distributed storage systems

    公开(公告)号:US09952923B2

    公开(公告)日:2018-04-24

    申请号:US15199128

    申请日:2016-06-30

    Applicant: VMware, Inc.

    CPC classification number: G06F11/1004 G06F3/0619 G06F3/064 G06F3/0683

    Abstract: Systems and techniques are described for transferring data. A described technique includes receiving a request to transmit a data block from a first data storage device to a second data storage device. An attempt to read the data block from the first data storage device is made. A media error resulting from the attempt to read the data block from the first data storage device is detected. In response to detecting the media error, a new data block is generated and includes mismatched checksum data that causes a checksum mismatched error when the new data block is accessed. The new data block is transmitted for storage at the second data storage device in place of the data block.

    Dynamic over-provisioning of space in a log-structured file system

    公开(公告)号:US11748139B2

    公开(公告)日:2023-09-05

    申请号:US16999569

    申请日:2020-08-21

    Applicant: VMware, Inc.

    Abstract: The present disclosure is related to methods, systems, and machine-readable media for log-structured file system management operations. An aggregate amount of over-provisioned computing resources assigned to a plurality of log-structured file system (LFS) objects that are allocated for performance of memory management operations associated with a virtual storage area network (vSAN) can be determined. A subset of LFS objects that are candidates for performance of a particular memory management operation based on an amount of over-provisioned computing resources consumed by one or more LFS objects among the plurality of LFS objects exceeding a resource consumption threshold associated with the determined amount of over-provisioned computing resources assigned to the plurality of LFS objects can be selected. The particular memory management operation using one or more of the candidate LFS objects can be performed.

    Deduplication-aware load balancing in distributed storage systems

    公开(公告)号:US11461027B2

    公开(公告)日:2022-10-04

    申请号:US15653249

    申请日:2017-07-18

    Applicant: VMware, Inc.

    Abstract: Techniques for enabling deduplication-aware load balancing in a distributed storage system are provided. In one set of embodiments, a node of the distributed storage system can receive an I/O (Input/Output) request pertaining to a data block of a storage object stored on a local storage component of the node. The node can further determine whether the I/O request requires insertion of a new entry into a deduplication hash table associated with the local storage component or deletion of an existing entry from the deduplication hash table. If the I/O request requires insertion of a new hash table entry, the node can add an identifier of the data block into a probabilistic data structure associated with the local storage component, where the probabilistic data structure is configured to maintain information regarding distinct data blocks that are likely present in the local storage component. Alternatively, if the I/O request requires deletion of an existing hash table entry, the node can remove the identifier of the data block from the probabilistic data structure.

    DYNAMIC OVER-PROVISIONING OF SPACE IN A LOG-STRUCTURED FILE SYSTEM

    公开(公告)号:US20220058043A1

    公开(公告)日:2022-02-24

    申请号:US16999569

    申请日:2020-08-21

    Applicant: VMware, Inc.

    Abstract: The present disclosure is related to methods, systems, and machine-readable media for log-structured file system management operations. An aggregate amount of over-provisioned computing resources assigned to a plurality of log-structured file system (LFS) objects that are allocated for performance of memory management operations associated with a virtual storage area network (vSAN) can be determined. A subset of LFS objects that are candidates for performance of a particular memory management operation based on an amount of over-provisioned computing resources consumed by one or more LFS objects among the plurality of LFS objects exceeding a resource consumption threshold associated with the determined amount of over-provisioned computing resources assigned to the plurality of LFS objects can be selected. The particular memory management operation using one or more of the candidate LFS objects can be performed.

    Translating high level requirements policies to distributed configurations

    公开(公告)号:US11018988B2

    公开(公告)日:2021-05-25

    申请号:US14469424

    申请日:2014-08-26

    Applicant: VMware Inc.

    Abstract: Embodiments of the disclosure provide techniques for partitioning a resource object into multiple resource components of a cluster of host computer nodes in a distributed resources system. The distributed resources system translates high-level policy requirements into a resource configuration that the system accommodates. The system determines an allocation based on the policy requirements and identifies resource configurations that are available. Upon selecting a resource configuration, the distributed resources system assigns the allocation and associated values to the selected configuration and publishes the new configuration to other host computer nodes in the cluster.

    END-TO-END CHECKSUM IN A MULTI-TENANT ENCRYPTION STORAGE SYSTEM

    公开(公告)号:US20190215152A1

    公开(公告)日:2019-07-11

    申请号:US15866185

    申请日:2018-01-09

    Applicant: VMware, Inc.

    Abstract: A multi-tenant storage system can store clear text data and associated clear text checksum received from a storage tenant using their associated cryptographic key (“cryptokey”). When the clear text data is compressible, cryptographic data (“cryptodata”) is generated from a concatenation of the clear text checksum and compressed clear text data using the cryptokey. A cryptographic checksum (“cryptochecksum”) is generated from the cryptodata. When the clear text data is uncompressible, cryptographic data (“cryptodata”) is generated by encrypting the clear text data using the cryptokey with an extra verification step to make sure the clear text checksum can be rebuilt during the read request. A cryptographic checksum (“cryptochecksum”) is generated from the cryptodata. The cryptodata and associated cryptochecksum are stored in the multi-tenant storage system, so that repairs to damaged cryptodata can be made using the associated cryptochecksum.

    Synchronizing replicas with media errors in distributed storage systems

    公开(公告)号:US10198312B2

    公开(公告)日:2019-02-05

    申请号:US15943293

    申请日:2018-04-02

    Applicant: VMware, Inc.

    Abstract: Systems and techniques are described for transferring data. A described technique includes receiving a request to transmit a data block from a first data storage device to a second data storage device. An attempt to read the data block from the first data storage device is made. A media error resulting from the attempt to read the data block from the first data storage device is detected. In response to detecting the media error, a new data block is generated and includes mismatched checksum data that causes a checksum mismatched error when the new data block is accessed. The new data block is transmitted for storage at the second data storage device in place of the data block.

Patent Agency Ranking