Dynamically scaling application and storage system functions based on a heterogeneous resource pool available for use by a distributed storage management system

    公开(公告)号:US12079242B2

    公开(公告)日:2024-09-03

    申请号:US18047774

    申请日:2022-10-19

    Applicant: NetApp, Inc.

    CPC classification number: G06F16/27

    Abstract: Systems and methods for scaling application and/or storage system functions of a distributed storage system based on a heterogeneous resource pool are provided. According to one embodiment, the distributed storage system has a composable, service-based architecture that provides scalability, resiliency, and load balancing. The distributed storage system includes a cluster of nodes each potentially having differing capabilities in terms of processing, memory, and/or storage. The distributed storage system takes advantage of different types of nodes by selectively instating appropriate services (e.g., file and volume services and/or block and storage management services) on the nodes based on their respective capabilities. Furthermore, disaggregation of these services, facilitated by interposing a frictionless layer (e.g., in the form of one or more globally accessible logical disks) therebetween, enables independent and on-demand scaling of either or both of application and storage system functions within the cluster while making use of the heterogeneous resource pool.

    Metadata compaction
    62.
    发明授权

    公开(公告)号:US12073123B2

    公开(公告)日:2024-08-27

    申请号:US17963077

    申请日:2022-10-10

    Applicant: NetApp Inc.

    CPC classification number: G06F3/0661 G06F3/0608 G06F3/064 G06F3/0673

    Abstract: Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object. Storing compacted indirect blocks into memory allows for more block numbers to be stored within memory. In this way, the block numbers are read from memory is faster than loading the block numbers from disk.

    FREEING PAGES WITHIN PERSISTENT MEMORY
    65.
    发明公开

    公开(公告)号:US20240086374A1

    公开(公告)日:2024-03-14

    申请号:US18514529

    申请日:2023-11-20

    Applicant: NetApp Inc.

    CPC classification number: G06F16/1805 G06F16/162 G06F16/2246

    Abstract: Techniques are provided for utilizing a log to free pages from persistent memory. A log is maintained to comprise a list of page block numbers of pages within persistent memory of a node to free. A page block number, of a page, within the log is identified for processing. A reference count, corresponding to a number of references to the page block number, is identified. In response to the reference count being greater than 1, the reference count is decremented and the page block number is removed from the log. In response to the reference count being 1, the page is freed from the persistent memory and the page block number is removed from the log.

    FILE SYSTEM FORMAT FOR PERSISTENT MEMORY
    67.
    发明公开

    公开(公告)号:US20230336183A1

    公开(公告)日:2023-10-19

    申请号:US18186657

    申请日:2023-03-20

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for implementing a file system format for persistent memory. A node, comprising persistent memory, receives an operation comprising a file identifier and file system instance information. A list of file system info objects are evaluated to identify a file system info object matching the file system instance information. An inofile, identified by the file system info object as being associated with inodes of files within an instance of the file system targeted by the operation, is traversed to identify an inode matching the file identifier. If the inode comprises an indicator that the file is tiered into the persistent memory, then the inode it utilized to facilitate execution of the operation upon the persistent memory. Otherwise, the operation is routed to a storage file system tier for execution by a storage file system upon storage associated with the node.

    Persistent memory file system reconciliation

    公开(公告)号:US11630733B2

    公开(公告)日:2023-04-18

    申请号:US17018176

    申请日:2020-09-11

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for persistent memory file system reconciliation. As part of the persistent memory file system reconciliation, high level file system metadata associated with a persistent memory file system of persistent memory is reconciled. Client access to the persistent memory file system is inaccessible until reconciliation of the high level file system metadata has completed. A first scanner is executed to traverse pages of the persistent memory in order to fix local inconsistencies associated with the pages. A local inconsistency of a first set of metadata or data of a page is fixed using a second set of metadata or data of the page. The first scanner is executed asynchronously in parallel with processing client I/O directed to the persistent memory file system.

    OBJECT AND SEQUENCE NUMBER MANAGEMENT

    公开(公告)号:US20230052732A1

    公开(公告)日:2023-02-16

    申请号:US17976946

    申请日:2022-10-31

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

Patent Agency Ranking