Scalable segment cleaning for a log-structured file system

    公开(公告)号:US11494110B2

    公开(公告)日:2022-11-08

    申请号:US16999897

    申请日:2020-08-21

    Applicant: VMware, Inc.

    Abstract: Scalable segment cleaning for log-structured file systems (LFSs) includes determining counts of segment cleaners and virtual nodes, with each virtual node being associated with a plurality of objects. Each virtual node is assigned to a selected segment cleaner. Based at least on the assignments, performing, for each virtual node, segment cleaning of the objects by the assigned segment cleaner. A portion, less than all, of the virtual nodes are reassigned to a newly selected segment cleaner based on a change of the count of the segment cleaners and/or a change of the count of the virtual nodes. Based at least on the reassignments, segment cleaning of the objects is performed, for each reassigned virtual node, by the reassigned segment cleaner. In some examples, the objects comprise virtual machine disks (VMDKs) and the segment cleaning uses a segment usage table (SUT) to track segment usage and identify segment cleaning candidates.

    Dynamic Base Disk Mirroring for Linked Clones

    公开(公告)号:US20220317883A1

    公开(公告)日:2022-10-06

    申请号:US17222621

    申请日:2021-04-05

    Applicant: VMware, Inc.

    Abstract: Techniques for implementing dynamic base disk mirroring for linked clones are provided. In one set of embodiments, a first node in a distributed storage system can monitor a congestion level of a base disk residing on the first node, where the base disk is shared by a plurality of linked clones. Upon determining that the congestion level exceeds a threshold, the first node can send, to a second node, a request to create a mirror of the base disk on that second node. Upon receiving an acknowledgement from the second node that the mirror has been successfully created, the first node can update a mirror set associated with the base disk to include an entry identifying the mirror. The first node can then communicate the updated mirror set to one or more other nodes.

    Shrinking segment cleaning algorithm in an object storage

    公开(公告)号:US11435935B2

    公开(公告)日:2022-09-06

    申请号:US17100663

    申请日:2020-11-20

    Applicant: VMware, Inc.

    Abstract: A method for cleaning an object storage having a plurality of segments is provided. Each segment includes an identifier through which the segment is accessed. The method identifies a first segment in the plurality of segments. The first segment includes a first identifier and a first size. The method determines that a utilization ratio for the first segment is below a threshold. As a result, the method generates a second segment from the first segment, such that the second segment includes a second identifier that is the same as the first identifier and a second size that is smaller than the first size. The method then writes the second segment to the object storage.

    EMBEDDED REFERENCE COUNTS FOR FILE CLONES

    公开(公告)号:US20220188267A1

    公开(公告)日:2022-06-16

    申请号:US17119959

    申请日:2020-12-11

    Applicant: VMware, Inc.

    Abstract: Techniques for efficiently managing a file clone from a filesystem which supports efficient volume snapshots are provided. In some embodiments, a system may receive an instruction to remove the file clone from the filesystem. The file clone may be a point-in-time copy of metadata of an original file. The system may further—for a file map entry in a filesystem tree associated with the file clone, the file map entry indicating a data block—decrement a reference count in a reference count entry associated with the file map entry. The reference count entry may be stored in the filesystem tree according to a key and the key may comprise an identification of the original file. The system may further reclaim the data block in a storage system when the reference count is zero.

    Efficient segment cleaning employing local copying of data blocks in log-structured file systems of distributed data systems

    公开(公告)号:US11334497B2

    公开(公告)日:2022-05-17

    申请号:US16914171

    申请日:2020-06-26

    Applicant: VMware, Inc.

    Abstract: Client data is structured as a set of data blocks. A first subset of data blocks is stored on a current segment of a plurality of disks. A second subset of data blocks is stored on a previous segment. A request to clean client data is received. The request includes a request to update the current segment to include the second subset of data blocks. The second subset of data blocks is accessed and transmitted from a lower layer to a higher layer of the system. Parity data is generated at the higher layer. The parity data is transmitted to the lower layer. The lower layer is employed to generate a local copy of the second subset of data blocks. Each local address that references the local copy of the second subset of data blocks is included in the current segment. The parity data is written in the current segment.

    Issuing efficient writes to erasure coded objects in a distributed storage system with two tiers of storage

    公开(公告)号:US11334277B2

    公开(公告)日:2022-05-17

    申请号:US16842649

    申请日:2020-04-07

    Applicant: VMware, Inc.

    Abstract: Techniques for issuing efficient writes to an erasure coded storage object in a distributed storage system are provided. In one set of embodiments, a node of the system can receive a write request for updating a logical data block of the storage object, write data/metadata for the block to a record in a data log of a metadata object of the storage object (where the metadata object is stored on a performance storage tier), place the block data in a free slot of an in-memory bank, and determine whether the in-memory bank has become full. If the in-memory bank is full, the node can further allocate a segment in a capacity object of the storage object for holding contents of the in-memory bank (where the capacity object is stored on a capacity storage tier), and write the in-memory bank contents via a full stripe write to the allocated segment.

    SCALABLE SEGMENT CLEANING FOR A LOG-STRUCTURED FILE SYSTEM

    公开(公告)号:US20220057955A1

    公开(公告)日:2022-02-24

    申请号:US16999897

    申请日:2020-08-21

    Applicant: VMware, Inc.

    Abstract: Scalable segment cleaning for log-structured file systems (LFSs) includes determining counts of segment cleaners and virtual nodes, with each virtual node being associated with a plurality of objects. Each virtual node is assigned to a selected segment cleaner. Based at least on the assignments, performing, for each virtual node, segment cleaning of the objects by the assigned segment cleaner. A portion, less than all, of the virtual nodes are reassigned to a newly selected segment cleaner based on a change of the count of the segment cleaners and/or a change of the count of the virtual nodes. Based at least on the reassignments, segment cleaning of the objects is performed, for each reassigned virtual node, by the reassigned segment cleaner. In some examples, the objects comprise virtual machine disks (VMDKs) and the segment cleaning uses a segment usage table (SUT) to track segment usage and identify segment cleaning candidates.

    REDUCED WRITE AMPLIFICATION USING SMALL-BLOCK LOG

    公开(公告)号:US20220012132A1

    公开(公告)日:2022-01-13

    申请号:US16925168

    申请日:2020-07-09

    Applicant: VMware, Inc.

    Abstract: Processing write requests from clients includes logging the associated data into logs corresponding to the data objects targeted by the write requests. The logs are persisted by combining log entries from each of the logs into one ore more fixed-size data blocks. The fixed-size data blocks are inserted into a data tree stored on a block-based storage device.

    EFFICIENT ACCESSING METHODS FOR BYPASSING SECOND LAYER MAPPING OF DATA BLOCKS IN FILE SYSTEMS OF DISTRIBUTED DATA SYSTEMS

    公开(公告)号:US20210382825A1

    公开(公告)日:2021-12-09

    申请号:US16914149

    申请日:2020-06-26

    Applicant: VMware, Inc.

    Abstract: Systems for accessing client data is described. A request to access a first data block is received. The request indicates a first logical address referencing the first data block. First mapping data is employed to identify a first physical addresses corresponding to the first logical addresses. The first mapping data encodes a first LOM transaction ID and candidate local addresses. The first mapping data is employed to identify the candidate local address and the first LOM transaction ID. A usage table is employed to determine the current status of the first LOM transaction ID. The candidate local address is employed to access the first data block. Second mapping data is employed to identify an updated local address of the set of local addresses. The updated local address currently references the first data block. The updated local address is employed to access the first data block.

Patent Agency Ranking