Synchronizing a stale component of a distributed object using a delta component during maintenance

    公开(公告)号:US11947827B2

    公开(公告)日:2024-04-02

    申请号:US16875624

    申请日:2020-05-15

    Applicant: VMware, Inc.

    CPC classification number: G06F3/065 G06F3/0617 G06F3/0689

    Abstract: The disclosure herein describes enhancing data durability of a base component using a delta component. A delta component is generated based on the base component becoming unavailable. The delta component is configured to include unwritten storage space with an address space matching the base component and a tracking bitmap associated with data blocks of the address space of the delta component. Write operations targeted for the base component are routed to the delta component. Based on the routed write operations, bits associated with data blocks affected by the write operations are changed in the tracking bitmap. Based on the base component becoming available, data blocks affected by routed write operations are identified based on the tracking bitmap and the identified data blocks are synchronized from the delta component to the base component. The delta component is then removed.

    EFFICIENT INCREMENTAL JOURNAL TRUNCATION POLICY

    公开(公告)号:US20240078010A1

    公开(公告)日:2024-03-07

    申请号:US17929204

    申请日:2022-09-01

    Applicant: VMware, Inc.

    CPC classification number: G06F3/0604 G06F3/0655 G06F3/0673

    Abstract: A method for efficient journal truncation is provided. A method for journal truncation includes maintaining a journal in a memory of a computing system including a plurality of records. Each record indicates a transaction in an ordered data structure. The method includes maintaining a truncation queue in the memory including one or more entries. Each entry in the truncation queue includes a physical on-disk offset associated with a different record of the plurality of records. The method includes determining to truncate the journal and truncating records, of the plurality of records, from the journal starting from a beginning record in the journal up to the record with the physical on-disk offset associated a least recent entry of the one or more entries in the truncation queue, where the truncating includes removing the records from the memory.

    Enhanced locking mechanism for B+ tree data structures

    公开(公告)号:US11663186B2

    公开(公告)日:2023-05-30

    申请号:US17177686

    申请日:2021-02-17

    Applicant: VMware, Inc.

    CPC classification number: G06F16/2246 G06F16/2343 G06F16/24552 G06F16/288

    Abstract: A method for modifying key-value pairs of a B+ tree is provided. The method receives a request to modify a particular key-value pair. Each node of the tree has a modification number. The method traverses a path on the tree from the root node toward the particular node. The traversing includes upon reaching a parent node of the path, acquiring a shared lock on both the parent node and a child node one level below the parent node. Upon determining that the child node is the particular node, the method stores the modification number of the particular node, releases the shared lock on the particular node, compares a current modification number of the node with its stored number, and acquires an exclusive lock on the node if the numbers are the same. The method increments the current modification number of the node and modifies it while in the exclusive lock.

    Enhanced data encryption in distributed datastores using random tweaks stored in data blocks

    公开(公告)号:US11573711B2

    公开(公告)日:2023-02-07

    申请号:US16827692

    申请日:2020-03-23

    Applicant: VMware, Inc.

    Abstract: A method for encrypting data in one or more data blocks is provided. The method receives a first data block to be written to a physical storage that includes one or more physical disks. The method applies a first random tweak to data indicative of the first data block to generate a first encrypted data block, and writes the first encrypted data block and the first random tweak to a first physical block of the physical storage. The method receives a second data block to be written to the physical storage. The method then applies a second random tweak, different than the first random tweak, to data indicative of the second data block to generate a second encrypted data block, and writes the second encrypted data block and the second random tweak to a second physical block of the physical storage.

    Performing resynchronization jobs in a distributed storage system based on a parallelism policy

    公开(公告)号:US11494083B2

    公开(公告)日:2022-11-08

    申请号:US16504204

    申请日:2019-07-05

    Applicant: VMWARE, INC.

    Abstract: The disclosure herein describes performing resynchronization (“resync”) jobs in a distributed storage system based on a parallelism policy. A resync job is obtained from a queue and input/output (I/O) resources that will be used during execution of the resync job are identified. Available bandwidth slots of each I/O resource of the identified I/O resources are determined. The parallelism policy is applied to the identified I/O resources and the available bandwidth slots. Based on the application of the parallelism policy, a bottleneck resource of the I/O resources is determined and a parallel I/O value is calculated based on the available bandwidth slots of the bottleneck resource, wherein the parallel I/O value indicates a quantity of I/O tasks that can be performed in parallel. The resync job is executed using the I/O resources, the execution of the resync job including performance of I/O tasks in parallel based on the parallel I/O value.

    System and methods of efficiently resyncing failed components without bitmap in an erasure-coded distributed object with log-structured disk layout

    公开(公告)号:US11429498B2

    公开(公告)日:2022-08-30

    申请号:US16870861

    申请日:2020-05-08

    Applicant: VMware, Inc.

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for resynchronizing data in a storage system. One of the methods includes determining that a particular disk of a capacity object of a storage system was offline for an interval of time, wherein the capacity object comprises a plurality of segments, and wherein the storage system comprises a segment usage table identifying a linked list of particular segments of the capacity object that are in use; determining a time point at which the particular disk went offline; determining one or more first segments of the capacity object that were modified after the time point, wherein determining one or more first segments comprises determining each segment of the segment usage table having a transaction ID that is larger than the time point; and resynchronizing, for each first segment, a portion of the particular disk corresponding to the first segment.

    Efficient accessing methods for bypassing second layer mapping of data blocks in file systems of distributed data systems

    公开(公告)号:US11263146B2

    公开(公告)日:2022-03-01

    申请号:US16914149

    申请日:2020-06-26

    Applicant: VMware, Inc.

    Abstract: Systems for accessing client data is described. A request to access a first data block is received. The request indicates a first logical address referencing the first data block. First mapping data is employed to identify a first physical addresses corresponding to the first logical addresses. The first mapping data encodes a first LOM transaction ID and candidate local addresses. The first mapping data is employed to identify the candidate local address and the first LOM transaction ID. A usage table is employed to determine the current status of the first LOM transaction ID. The candidate local address is employed to access the first data block. Second mapping data is employed to identify an updated local address of the set of local addresses. The updated local address currently references the first data block. The updated local address is employed to access the first data block.

    Resumable replica resynchronization

    公开(公告)号:US11157195B2

    公开(公告)日:2021-10-26

    申请号:US16445649

    申请日:2019-06-19

    Applicant: VMware, Inc.

    Abstract: Systems and techniques are described for transferring data. A described technique includes determining that a first replica of an object stored at a first host has become available to a distributed storage system after previously being unavailable to the distributed storage system. The object includes a range of memory addresses at which data of the object is stored. In response to determining that the first replica has become available, resyncing data for the first replica is obtained. The resyncing data indicates whether each range of memory addresses is synchronized at the first replica with other replicas of the object. Tracking data for the first replica is obtained. The tracking data indicates whether data stored at the range of memory addresses of the object has been modified at a second replica while the first replica was unavailable. The resyncing data is updated based on the tracking data.

    Issuing Efficient Writes to Erasure Coded Objects in a Distributed Storage System with Two Tiers of Storage

    公开(公告)号:US20210311653A1

    公开(公告)日:2021-10-07

    申请号:US16842649

    申请日:2020-04-07

    Applicant: VMware, Inc.

    Abstract: Techniques for issuing efficient writes to an erasure coded storage object in a distributed storage system are provided. In one set of embodiments, a node of the system can receive a write request for updating a logical data block of the storage object, write data/metadata for the block to a record in a data log of a metadata object of the storage object (where the metadata object is stored on a performance storage tier), place the block data in a free slot of an in-memory bank, and determine whether the in-memory bank has become full. If the in-memory bank is full, the node can further allocate a segment in a capacity object of the storage object for holding contents of the in-memory bank (where the capacity object is stored on a capacity storage tier), and write the in-memory bank contents via a full stripe write to the allocated segment.

Patent Agency Ranking