SUPPORTING STORAGE USING A MULTI-WRITER LOG-STRUCTURED FILE SYSTEM

    公开(公告)号:US20210318992A1

    公开(公告)日:2021-10-14

    申请号:US16847543

    申请日:2020-04-13

    Applicant: VMware, Inc.

    Abstract: Solutions for supporting storage using a multi-writer log-structured file system (LFS) are disclosed that include receiving incoming data from an object of a plurality of objects that are configured to simultaneously write to the LFS from different nodes; based at least on receiving the incoming data, determining whether sufficient free segments are available in a local segment usage table (SUT) for writing the incoming data; based at least on determining that insufficient free segments are available , requesting allocation of new free segments; writing the incoming data to a log; acknowledging the writing to the object; determining whether the log has accumulated a full segment of data; based at least on determining that the log has accumulated a full segment of data, writing the full segment of data to a first segment of the free segments; and updating the local SUT to mark the first segment as no longer free.

    FAST ALGORITHM TO FIND FILE SYSTEM DIFFERENCE FOR DEDUPLICATION

    公开(公告)号:US20210064580A1

    公开(公告)日:2021-03-04

    申请号:US16552965

    申请日:2019-08-27

    Applicant: VMware, Inc.

    Abstract: The disclosure provides techniques for deduplicating files. The techniques include, upon creating or modifying a file, placing a logical timestamp of the current logical time, within a queue associated with the directory of the file. The techniques further include placing the logical timestamp within a queue of each parent directory of the directory of the file. To determine a set of files for deduplication, the techniques disclosed herein identify files that have been modified within a logical time range. The set of files modified within a logical time is identified by traversing directories of a storage system, the directories being organized within a tree structure. If a directory's queue does not contain a timestamp that is within the logical time range, then all child directories can be skipped over for further processing, such that no files within the child directories end up being within the set of files for deduplication.

    SUPPORTING FILE SYSTEM CLONES IN ANY ORDERED KEY-VALUE STORE

    公开(公告)号:US20180364950A1

    公开(公告)日:2018-12-20

    申请号:US15628016

    申请日:2017-06-20

    Applicant: VMware, Inc.

    Abstract: Embodiments described herein are related to cloning a volume in a file system. In some embodiments, a directory hard link is used to generate a clone of the root node of the volume. In certain embodiments, upon determining that a file or directory of the clone which comprises a hard link to an index node has been modified, a new object directory is generated beneath a root node of the volume. The index node may be added to the new object directory and one or more files and directories in the volume which link to the index node may be updated to contain symbolic links to the index node in the new object directory. In certain embodiments, a copy-on-write operation is performed in order to copy the file or directory and the new object directory to the clone.

    ABORTABLE TRANSACTIONS USING VERSIONED TUPLE CACHE
    34.
    发明申请
    ABORTABLE TRANSACTIONS USING VERSIONED TUPLE CACHE 审中-公开
    使用VERSIONED TUPLE CACHE的可移动交易

    公开(公告)号:US20160378815A1

    公开(公告)日:2016-12-29

    申请号:US14752175

    申请日:2015-06-26

    Applicant: VMware, Inc.

    Abstract: A transaction manager for handling operations on data in a storage system provides a system for executing transactions that uses a versioned tuple cache to achieve fast, abortable transactions using a redo-only log. The transaction manager updates an in-memory key-value store and also attaches a transaction identifier to the tuple as a minor key. Opportunistic locking can be accomplished due to the low cost of aborting transactions.

    Abstract translation: 用于处理存储系统中的数据操作的事务管理器提供用于执行使用版本化的元组高速缓存以使用仅重做日志来实现快速,可中止事务的事务的系统。 事务管理器更新内存中键值存储,并将事务标识符附加到元组作为次要密钥。 由于中止交易的成本低,可以实现机会锁定。

    MAINTAINING HOLE BOUNDARY INFORMATION FOR RESTORING SNAPSHOTS FROM PARITY
    35.
    发明申请
    MAINTAINING HOLE BOUNDARY INFORMATION FOR RESTORING SNAPSHOTS FROM PARITY 审中-公开
    维护从边界恢复活动的边界信息

    公开(公告)号:US20160342468A1

    公开(公告)日:2016-11-24

    申请号:US14716740

    申请日:2015-05-19

    Applicant: VMware, Inc.

    Abstract: Exemplary methods, apparatuses, and systems maintain hole boundary information by calculating a block attribute parity value. For example, a request is received to write to a first block of a stripe of data. A block attribute of a second block is determined. The block attribute of the second block indicates whether the second block includes written data or is a hole. A block attribute parity value is calculated based upon both the block attribute of the first block and the block attribute of the second block. The block attribute of the first block indicates the first block includes written data based upon the received request. The block attribute parity value and the data parity value are stored on one of the physical storage devices in response to the received write request. As a result, if a disk is lost, holes can be recovered using the block attribute parity value.

    Abstract translation: 示例性方法,装置和系统通过计算块属性奇偶校验值来维持空穴边界信息。 例如,接收到写入数据条带的第一块的请求。 确定第二块的块属性。 第二块的块属性指示第二块是否包括写入数据或是孔。 基于第一块的块属性和第二块的块属性两者来计算块属性奇偶校验值。 第一块的块属性指示第一块包括基于接收到的请求的写入数据。 响应于接收到的写请求,块属性奇偶校验值和数据奇偶校验值被存储在物理存储设备之一上。 因此,如果磁盘丢失,则可以使用块属性奇偶校验值恢复孔。

    SCALABLE SPACE MANAGEMENT USING BITMAP SUMMARY TREE
    36.
    发明申请
    SCALABLE SPACE MANAGEMENT USING BITMAP SUMMARY TREE 审中-公开
    使用BITMAP总结树的可扩展空间管理

    公开(公告)号:US20160334997A1

    公开(公告)日:2016-11-17

    申请号:US14713698

    申请日:2015-05-15

    Applicant: VMware, Inc.

    Abstract: Systems and methods for allocating space in persistent storage are provided. A modified bitmap and a tree of bitmap summary pages are used to manage the free space of a large scale storage system. The bitmap is separated into fixed size pages and has bitmap summary entries to summarize the information in the bitmap. Bitmap summary pages can be further summarized into secondary summary pages. The tree data structure can continue to N levels until a topmost level has one bitmap summary page.

    Abstract translation: 提供了在持久存储器中分配空间的系统和方法。 使用修改后的位图和位图摘要页面树来管理大型存储系统的可用空间。 位图被分成固定大小的页面,并且具有位图摘要条目以总结位图中的信息。 位图摘要页面可以进一步汇总到次要摘要页面中。 树数据结构可以继续N级,直到最顶层具有一个位图摘要页。

    GARBAGE COLLECTION IN A LOG-STRUCTURED FILE SYSTEM

    公开(公告)号:US20250028679A1

    公开(公告)日:2025-01-23

    申请号:US18356129

    申请日:2023-07-20

    Applicant: VMware, Inc.

    Abstract: An example method of managing a log-structured file system (LFS) on a storage device includes: receiving, at storage software executing on a host, an operation that overwrites a data block, the data block included in a segment of the LFS; determining from first metadata stored on the storage device, a change in utilization of the segment from a first utilization value to a second utilization value; modifying second metadata stored on the storage device to change a relation between the segment and a first bucket to be a relation between the segment and a second bucket, the first utilization value included in a range of the first bucket and the second utilization value included in a range of the second bucket; and executing a garbage collection process for the LFS that uses the second metadata to identify for garbage collection a set of segments in the second bucket.

    DEDUPLICATING DATA CHUNKS USING CHUNK OBJECTS

    公开(公告)号:US20240419583A1

    公开(公告)日:2024-12-19

    申请号:US18333986

    申请日:2023-06-13

    Applicant: VMware, Inc.

    Abstract: The disclosure herein describes deduplicating data chunks using chunk objects. A batch of data chunks is obtained from an original data object and a hash value is calculated for each data chunk. A first duplicate data chunk is identified using the hash value and a hash map. A chunk logical block address (LBA) of a chunk object is assigned to the duplicate data chunk. Payload data of the duplicate data chunk is migrated from the original data object to the chunk object, and a chunk map is updated to map the chunk LBA to a physical sector address (PSA) of the migrated payload data on the chunk object. A hash entry is updated to map to the chunk object and the chunk LBA. An address map of the original data object is updated to map an LBA of the duplicate data chunk to the chunk object and the chunk LBA.

    BACKGROUND DELETION OF LARGE DIRECTORIES
    39.
    发明公开

    公开(公告)号:US20240111722A1

    公开(公告)日:2024-04-04

    申请号:US18057384

    申请日:2022-11-21

    Applicant: VMware, Inc.

    CPC classification number: G06F16/162 G06F16/185 G06F16/188

    Abstract: Deleting directories in a virtual distributed file system (VDFS), and non-virtual file systems, involves changing the name of a selected directory to a unique object identifier (UID) and moving the selected directory, named according to the UID, to a deletion target directory. A recursive process, implemented using a background deletion thread, starts in the current directory and identifies objects in the current directory. For an object that is a file or an empty directory, the object is added to a deletion queue. For an object that is a directory that is not empty, the recursion drops down into that directory as the new current directory. When the recursion has exhausted the selected directory, or some maximum object count has been reached, the objects identified in the deletion queue are deleted. This approach can also be used for file operations other than deletion, such as compression, encryption, and hashing.

    DYNAMIC GROWTH OF DATA CACHES USING BACKGROUND PROCESSES FOR HASH BUCKET GROWTH

    公开(公告)号:US20240070080A1

    公开(公告)日:2024-02-29

    申请号:US17900642

    申请日:2022-08-31

    Applicant: VMware, Inc.

    CPC classification number: G06F12/0864 G06F2212/1016 G06F2212/604

    Abstract: The disclosure describes growing a data cache using a background hash bucket growth process. A first memory portion is allocated to the data buffer of the data cache and a second memory portion is allocated to the metadata buffer of the data cache based on the cache growth instruction. The quantity of hash buckets in the hash bucket buffer is increased and the background hash bucket growth process is initiated, wherein the process is configured to rehash hash bucket entries of the hash bucket buffer in the increased quantity of hash buckets. A data entry is stored in the data buffer using the allocated first memory portion of the data cache and metadata associated with the data entry is stored using the allocated second memory portion of the metadata buffer, wherein a hash bucket entry associated with the data entry is stored in the increased quantity of hash buckets.

Patent Agency Ranking