Distributed File System that Provides Scalability and Resiliency

    公开(公告)号:US20220391359A1

    公开(公告)日:2022-12-08

    申请号:US17449758

    申请日:2021-10-01

    Applicant: NetApp, Inc.

    Abstract: A distributed storage management system comprising nodes that form a cluster, a distributed block layer that spans the nodes in the cluster, and file system instances deployed on the nodes. Each file system instance comprises a data management subsystem and a storage management subsystem disaggregated from the data management subsystem. The storage management subsystem comprises a node block store that forms a portion of the distributed block layer and a storage manager that manages a key-value store and virtualized storage supporting the node block store. A file system volume hosted by the data management subsystem maps to a logical block device hosted by the virtualized storage in the storage management subsystem. The key-value store includes, for a data block of the logical block device, a key that comprises a block identifier for the logical block device and a value that comprises the data block.

    Metadata compaction
    142.
    发明授权

    公开(公告)号:US11467774B2

    公开(公告)日:2022-10-11

    申请号:US17083945

    申请日:2020-10-29

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object.

    Freeing pages within persistent memory

    公开(公告)号:US11461281B2

    公开(公告)日:2022-10-04

    申请号:US16852584

    申请日:2020-04-20

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for utilizing a log to free pages from persistent memory. A log is maintained to comprise a list of page block numbers of pages within persistent memory of a node to free. A page block number, of a page, within the log is identified for processing. A reference count, corresponding to a number of references to the page block number, is identified. In response to the reference count being greater than 1, the reference count is decremented and the page block number is removed from the log. In response to the reference count being 1, the page is freed from the persistent memory and the page block number is removed from the log.

    LOW-OVERHEAD ATOMIC WRITES FOR PERSISTENT MEMORY

    公开(公告)号:US20220300429A1

    公开(公告)日:2022-09-22

    申请号:US17750707

    申请日:2022-05-23

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for atomic writes for persistent memory. In response to receiving a write operation, a new per-page structure with a new page block number is allocated. New data of the write operation is persisted to a new page of the persistent memory having the new page block number, and the new per-page structure is persisted to the persistent memory. If the write operation targets a hole after the new data and the new per-page structure have been persisted, then a new per-page structure identifier of the new per-page structure is inserted into a parent indirect page of a page comprising the new data. If the write operation targets old data after the new data and the new per-page structure have been persisted, then an old per-page structure of the old data is updated with the new page block number.

    Network Storage Failover Systems and Associated Methods

    公开(公告)号:US20220283915A1

    公开(公告)日:2022-09-08

    申请号:US17751944

    申请日:2022-05-24

    Applicant: NETAPP, INC.

    Abstract: Failover methods and systems for a networked storage environment are provided. In one aspect, a read request associated with a first storage object is received, during a replay of entries of a log stored in a non-volatile memory of a second storage node for a failover operation initiated in response to a failure at a first storage node. The second storage node operates as a partner node of the first storage node. The read request is processed using a filtering data structure that is generated from the log prior to the replay and identifies each log entry. The read request is processed when the log does not have an entry associated with the read request, and when the filtering data structure includes an entry associated with the read request, the requested data is located at the non-volatile memory.

    Maintaining and recomputing reference counts in a persistent memory file system

    公开(公告)号:US11422903B2

    公开(公告)日:2022-08-23

    申请号:US16944001

    申请日:2020-07-30

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for maintaining and recomputing reference counts in a persistent memory file system of a node. Primary reference counts are maintained for pages within persistent memory of the node. In response to receiving a first operation to link a page into a persistent memory file system of the persistent memory, a primary reference count of the page is incremented before linking the page into the persistent memory file system. In response to receiving a second operation to unlink the page from the persistent memory file system, the page is unlinked from the persistent memory file system before the primary reference count is decremented. Upon the node recovering from a crash, the persistent memory file system is traversed in order to update shadow reference counts for the pages with correct reference count values, which are used to overwrite the primary reference counts with the correct reference count values.

    Co-located journaling and data storage for write requests

    公开(公告)号:US11409457B1

    公开(公告)日:2022-08-09

    申请号:US17239189

    申请日:2021-04-23

    Applicant: NetApp, Inc.

    Abstract: A method and system for co-locating journaling and data storage based on write requests. A write request that includes metadata and data is received from a client. A logical storage unit for storing the metadata and the data is identified. The logical storage unit is divided into a journal partition and a volume partition. The journal partition includes a first log and a second log. Which of the first log and the second log is an active log and which of the first log and the second log is an inactive log are identified. The metadata is recorded in a first location in the active log and the data is recorded in a second location in the active log during a single I/O operation. A reply is sent to the client after the metadata and the data are recorded in the journal partition.

    Low-overhead atomic writes for persistent memory

    公开(公告)号:US11341056B2

    公开(公告)日:2022-05-24

    申请号:US16852589

    申请日:2020-04-20

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for atomic writes for persistent memory. In response to receiving a write operation, a new per-page structure with a new page block number is allocated. New data of the write operation is persisted to a new page of the persistent memory having the new page block number, and the new per-page structure is persisted to the persistent memory. If the write operation targets a hole after the new data and the new per-page structure have been persisted, then a new per-page structure identifier of the new per-page structure is inserted into a parent indirect page of a page comprising the new data. If the write operation targets old data after the new data and the new per-page structure have been persisted, then an old per-page structure of the old data is updated with the new page block number.

Patent Agency Ranking