Distributed file system consistency check

    公开(公告)号:US10417193B2

    公开(公告)日:2019-09-17

    申请号:US15163471

    申请日:2016-05-24

    Applicant: VMware, Inc.

    Abstract: Exemplary methods, apparatuses, and systems include a controller node receiving a request to perform a consistency check of a distributed file system. The controller node transmits, to each of a plurality of nodes, a request for the node to use logical metadata of the distributed file system owned by the node to construct an expected copy of physical metadata mapped to the logical metadata, determine which of the plurality of nodes own actual portions of the physical metadata, transmit corresponding portions of the expected copy of the physical metadata to each of the nodes determined to own actual portions of the physical metadata, and compare expected copies of the physical metadata received from other nodes to the actual physical metadata owned by the node. The controller node receives a result of the comparison from each of the nodes, aggregates the received results, and generates an error report.

    SYSTEM AND METHOD FOR SPEED UP DATA REBUILD IN A DISTRIBUTED STORAGE SYSTEM WITH LOCAL DEDUPLICATION

    公开(公告)号:US20190227727A1

    公开(公告)日:2019-07-25

    申请号:US15880391

    申请日:2018-01-25

    Applicant: VMware, Inc.

    Abstract: A method includes obtaining a plurality of representations corresponding respectively to a plurality of blocks of data stored on a source node. A plurality of data pairs are sent to a destination node, where each data pair includes a logical address associated with a block of data from the plurality of blocks of data and the corresponding representation of the block of data. A determination is made whether the blocks of data associated with the respective logical addresses are duplicates of data stored on the destination node. In accordance with an affirmative determination, a reference to a physical address of the block of data stored on the destination node is stored. In accordance with a negative determination, an indication that the data corresponding to the respective logical address is not a duplicate is stored. The data indicated as not being a duplicate is written to the destination node.

    Shared data cache for kernel bypass applications

    公开(公告)号:US10235298B2

    公开(公告)日:2019-03-19

    申请号:US15622014

    申请日:2017-06-13

    Applicant: VMware, Inc.

    Inventor: Wenguang Wang

    Abstract: Techniques for implementing a shared data cache for kernel bypass applications are provided. In one set of embodiments, a shared data caching (SDC) service associated with an instance of a kernel bypass application can create a named shared memory region in user space, where the kernel bypass application is configured to use a user-level Input/Output (I/O) stack for accessing a physical I/O device. The SDC service can further map the named shared memory region into a virtual memory address space of the instance. Then, at a time the instance issues an I/O read request to the physical I/O device, the SDC service can process the I/O read request by accessing the named shared memory region as a data cache.

    MANAGEMENT OF B-TREE LEAF NODES WITH VARIABLE SIZE VALUES

    公开(公告)号:US20180373727A1

    公开(公告)日:2018-12-27

    申请号:US15633726

    申请日:2017-06-26

    Applicant: VMware, Inc.

    Abstract: System and method for managing leaf nodes of a B-tree for a file system of a computer system utilize used slots in a directory section of a leaf node to index variable size key-value pair entries stored in a data section of the leaf node and free spaces slots in the directory section to index contiguous free spaces in the data section. Contents of the free space slots in the directory section are updated in response to changes in the contiguous free spaces in the data section to manage free space in the data section of the leaf node.

    Space efficient persistence of an in-memory table

    公开(公告)号:US09892004B2

    公开(公告)日:2018-02-13

    申请号:US14857845

    申请日:2015-09-18

    Applicant: VMWARE, INC.

    Abstract: Techniques for efficiently storing the state of an in-memory table to persistent storage are described. In one embodiment, one or more requests to update an entry in an in-memory table with one or more values are received, wherein the in-memory table is stored in non-persistent memory. The one or more entries in the in-memory table are then updated with the one or more values and one or more recent redo log entries that correspond to the one or more entries and one or more values are generated. One or more historical entries in the table are selected and one or more historical redo log entries that correspond to the one or more historical entries are generated. The recent redo log entry and the one or more historical redo log entries are saved to a bounded redo log, wherein the bounded redo log is stored in persistent storage.

    DISTRIBUTED FILE SYSTEM CONSISTENCY CHECK
    126.
    发明申请

    公开(公告)号:US20170344597A1

    公开(公告)日:2017-11-30

    申请号:US15163471

    申请日:2016-05-24

    Applicant: VMware, Inc.

    CPC classification number: G06F16/182 G06F16/184

    Abstract: Exemplary methods, apparatuses, and systems include a controller node receiving a request to perform a consistency check of a distributed file system. The controller node transmits, to each of a plurality of nodes, a request for the node to use logical metadata of the distributed file system owned by the node to construct an expected copy of physical metadata mapped to the logical metadata, determine which of the plurality of nodes own actual portions of the physical metadata, transmit corresponding portions of the expected copy of the physical metadata to each of the nodes determined to own actual portions of the physical metadata, and compare expected copies of the physical metadata received from other nodes to the actual physical metadata owned by the node. The controller node receives a result of the comparison from each of the nodes, aggregates the received results, and generates an error report.

    DATA DEDUPLICATION IN A BLOCK-BASED STORAGE SYSTEM
    128.
    发明申请
    DATA DEDUPLICATION IN A BLOCK-BASED STORAGE SYSTEM 审中-公开
    基于块的存储系统中的数据校验

    公开(公告)号:US20160350325A1

    公开(公告)日:2016-12-01

    申请号:US14724392

    申请日:2015-05-28

    Applicant: VMware, Inc.

    Abstract: A method and system are disclosed for storing client data objects in a deduplicated storage system. Deduplicated data may be stored in a plurality of physical data blocks. A content map layer can provide a mapping between the physical data blocks to logical map objects associated with the client data objects. The deduplicated data may be mapped to logical data blocks that comprise the client data objects.

    Abstract translation: 公开了一种用于将客户端数据对象存储在重复数据删除的存储系统中的方法和系统。 重复数据可以存储在多个物理数据块中。 内容映射图层可以提供物理数据块与与客户端数据对象相关联的逻辑映射对象之间的映射。 重复数据删除的数据可以映射到构成客户数据对象的逻辑数据块。

    Efficient unmap handling using synchronized redundant components of data objects

    公开(公告)号:US12124701B2

    公开(公告)日:2024-10-22

    申请号:US18159656

    申请日:2023-01-25

    Applicant: VMware, Inc.

    CPC classification number: G06F3/061 G06F3/0652 G06F3/0679

    Abstract: Components of a distributed data object are synchronized using streamlined tracking metadata. A target component of the distributed data object is detected as it becomes available and stale. A source component that is up-to-date and that mirrors the address space of the detected target component is identified. A set of mapped address ranges and a set of unmapped address ranges of the identified source component are obtained. A mapped address range of the target component that corresponds with an unmapped address range of the source component is identified. The identified mapped address range of the target component is then synchronized with the corresponding unmapped address range of the source component. Thus, unmapped address ranges are synchronized without using tracking metadata of the source component.

Patent Agency Ranking