公开(公告)号：US11157372B2

公开(公告)日：2021-10-26

申请号：US16394357

申请日：2019-04-25

Applicant: NetApp, Inc.

Inventor： Christopher Cason

IPC: G06F12/02 ,  G06F12/06 ,  G06F11/14 ,  G06F16/18

Abstract: A technique is configured to reduce an amount of memory (i.e., memory footprint) usage by each storage node of a cluster needed to store metadata while providing fast and efficient servicing of data in accordance with storage requests issued by a client of the cluster. Illustratively, a block identifier (ID) is used to identify a block of data serviced by the storage node. Metadata embodied as mappings between block IDs and locations of data blocks in the cluster are illustratively maintained in map fragments. A map fragment may be embodied as “active” map fragment or a “frozen” map fragment. An active map fragment refers to a map fragment that has space available to store a mapping, whereas a frozen map fragment refers to a map fragment that is full, i.e., has no available space for storing a mapping. In order to reduce the memory footprint of each storage node, yet still provide fast and efficient servicing of data by the node, the active map fragments are preferably maintained in memory as “in-core” data structures, whereas the frozen map fragments are paged-out and stored on storage devices of the cluster as “on-disk” map fragment structures.

公开(公告)号：US20230367746A1

公开(公告)日：2023-11-16

申请号：US18359192

申请日：2023-07-26

Applicant: NetApp, Inc.

Inventor： Ravikanth Dronamraju ,  Ananthan Subramanian ,  Daniel McCarthy ,  Christopher Cason ,  Arindam Banerjee

IPC: G06F16/188 ,  G06F16/182 ,  G06F9/50

CPC classification number: G06F16/188 ,  G06F16/182 ,  G06F9/5077

Abstract: In various examples, data storage is managed using a distributed storage management system that is resilient. Data blocks of a logical block device may be distributed across multiple nodes in a cluster. The logical block device may correspond to a file system volume associated with a file system instance deployed on a selected node within a distributed block layer of a distributed file system. Each data block may have a location in the cluster identified by a block identifier associated with each data block. Each data block may be replicated on at least one other node in the cluster. A metadata object corresponding to a logical block device that maps to the file system volume may be replicated on at least another node in the cluster. Each data block and the metadata object may be hosted on virtualized storage that is protected using redundant array independent disks (RAID).

公开(公告)号：US20230121460A1

公开(公告)日：2023-04-20

申请号：US18047774

申请日：2022-10-19

Applicant: NetApp, Inc.

Inventor： Arindam Banerjee ,  Daniel McCarthy ,  Christopher Cason ,  Ananthan Subramanian

IPC: G06F16/27

Abstract: Systems and methods for scaling application and/or storage system functions of a distributed storage system based on a heterogeneous resource pool are provided. According to one embodiment, the distributed storage system has a composable, service-based architecture that provides scalability, resiliency, and load balancing. The distributed storage system includes a cluster of nodes each potentially having differing capabilities in terms of processing, memory, and/or storage. The distributed storage system takes advantage of different types of nodes by selectively instating appropriate services (e.g., file and volume services and/or block and storage management services) on the nodes based on their respective capabilities. Furthermore, disaggregation of these services, facilitated by interposing a frictionless layer (e.g., in the form of one or more globally accessible logical disks) therebetween, enables independent and on-demand scaling of either or both of application and storage system functions within the cluster while making use of the heterogeneous resource pool.

公开(公告)号：US11627097B2

公开(公告)日：2023-04-11

申请号：US17187336

申请日：2021-02-26

Applicant: NetApp, Inc.

Inventor： Austino Longo ,  Christopher Cason ,  Daniel McCarthy

IPC: H04L47/80 ,  H04L43/0876 ,  H04L67/1097 ,  H04L67/125

Abstract: Systems and methods for quality of service management are provided. According to one embodiment, a non-transitory computer-readable medium comprises instructions that when executed by the processing resource cause the processing resource to receive, in a normalizing agent, one or more compute load parameters from one or more background compute processes executing on the one or more computer systems and one or more Quality of Service (QoS) parameters for one or more client compute processes executing on the one or more computer systems, convert the one or more compute load parameters to one or more normalized utilization metrics, and execute instructions that cause a processor to adjust a compute resource allocation dedicated to the one or more background compute processes based at least in part on the one or more normalized utilization metrics and the one or more QoS parameters.

公开(公告)号：US20220391359A1

公开(公告)日：2022-12-08

申请号：US17449758

申请日：2021-10-01

Applicant: NetApp, Inc.

Inventor： Ravikanth Dronamraju ,  Ananthan Subramanian ,  Daniel McCarthy ,  Christopher Cason ,  Arindam Banerjee

IPC: G06F16/188 ,  G06F16/182 ,  G06F9/50

Abstract: A distributed storage management system comprising nodes that form a cluster, a distributed block layer that spans the nodes in the cluster, and file system instances deployed on the nodes. Each file system instance comprises a data management subsystem and a storage management subsystem disaggregated from the data management subsystem. The storage management subsystem comprises a node block store that forms a portion of the distributed block layer and a storage manager that manages a key-value store and virtualized storage supporting the node block store. A file system volume hosted by the data management subsystem maps to a logical block device hosted by the virtualized storage in the storage management subsystem. The key-value store includes, for a data block of the logical block device, a key that comprises a block identifier for the logical block device and a value that comprises the data block.

公开(公告)号：US10949312B2

公开(公告)日：2021-03-16

申请号：US16394642

申请日：2019-04-25

Applicant: NetApp, Inc.

Inventor： Christopher Cason

IPC: G06F11/00 ,  G06F11/14 ,  G06F16/18 ,  G06F12/02 ,  G06F12/06

Abstract: A technique is configured to log and update metadata in a log-structured file system to facilitate recovery and restart in response to failure of a storage node of a cluster. A block identifier (ID) is used to identify a block of data serviced by the storage node. Metadata embodied as mappings between block IDs and locations of data blocks in the cluster are illustratively maintained in “active” and “frozen” map fragments. An active map fragment refers to a map fragment that has space available to store a mapping, whereas a frozen map fragment refers to a map fragment that no available space for storing a mapping. The active map fragments are maintained in memory as “in-core” data structures, whereas the frozen map fragments are paged-out and stored on storage devices of the cluster as “on-disk” map fragment structures. Each frozen map fragment written to a segment includes a pointer to a last written frozen map fragment to form a chain (e.g., linked-list) of on-disk frozen map fragments. Each time a data block is persisted on a segment of the storage devices, an active map fragment is populated in-core and a metadata write marker is recorded on the segment (on-disk) indicating the location of the data block that was written to the segment. If a storage node crashes when the active map fragment is only partially populated, the metadata write markers facilitate rebuild of the active map fragment upon recovery and restart of a storage service of the node.

公开(公告)号：US20250013614A1

公开(公告)日：2025-01-09

申请号：US18780353

申请日：2024-07-22

Applicant: NetApp, Inc.

Inventor： Ravikanth Dronamraju ,  Ananthan Subramanian ,  Daniel McCarthy ,  Christopher Cason ,  Arindam Banerjee

IPC: G06F16/188 ,  G06F9/50 ,  G06F16/182

Abstract: A distributed storage management system comprising nodes that form a cluster, a distributed block layer that spans the nodes in the cluster, and file system instances deployed on the nodes. Each file system instance comprises a data management subsystem and a storage management subsystem disaggregated from the data management subsystem. The storage management subsystem comprises a node block store that forms a portion of the distributed block layer and a storage manager that manages a key-value store and virtualized storage supporting the node block store. A file system volume hosted by the data management subsystem maps to a logical block device hosted by the virtualized storage in the storage management subsystem. The key-value store includes, for a data block of the logical block device, a key that comprises a block identifier for the logical block device and a value that comprises the data block.

公开(公告)号：US12045207B2

公开(公告)日：2024-07-23

申请号：US17449758

申请日：2021-10-01

Applicant: NetApp, Inc.

Inventor： Ravikanth Dronamraju ,  Ananthan Subramanian ,  Daniel McCarthy ,  Christopher Cason ,  Arindam Banerjee

IPC: G06F16/18 ,  G06F9/50 ,  G06F16/182 ,  G06F16/188

CPC classification number: G06F16/188 ,  G06F9/5077 ,  G06F16/182

Abstract: A distributed storage management system comprising nodes that form a cluster, a distributed block layer that spans the nodes in the cluster, and file system instances deployed on the nodes. Each file system instance comprises a data management subsystem and a storage management subsystem disaggregated from the data management subsystem. The storage management subsystem comprises a node block store that forms a portion of the distributed block layer and a storage manager that manages a key-value store and virtualized storage supporting the node block store. A file system volume hosted by the data management subsystem maps to a logical block device hosted by the virtualized storage in the storage management subsystem. The key-value store includes, for a data block of the logical block device, a key that comprises a block identifier for the logical block device and a value that comprises the data block.

公开(公告)号：US12038886B2

公开(公告)日：2024-07-16

申请号：US18359192

申请日：2023-07-26

Applicant: NetApp, Inc.

Inventor： Ravikanth Dronamraju ,  Ananthan Subramanian ,  Daniel McCarthy ,  Christopher Cason ,  Arindam Banerjee

IPC: G06F16/18 ,  G06F9/50 ,  G06F16/182 ,  G06F16/188

CPC classification number: G06F16/188 ,  G06F9/5077 ,  G06F16/182

Abstract: In various examples, data storage is managed using a distributed storage management system that is resilient. Data blocks of a logical block device may be distributed across multiple nodes in a cluster. The logical block device may correspond to a file system volume associated with a file system instance deployed on a selected node within a distributed block layer of a distributed file system. Each data block may have a location in the cluster identified by a block identifier associated with each data block. Each data block may be replicated on at least one other node in the cluster. A metadata object corresponding to a logical block device that maps to the file system volume may be replicated on at least another node in the cluster. Each data block and the metadata object may be hosted on virtualized storage that is protected using redundant array independent disks (RAID).

公开(公告)号：US20220278943A1

公开(公告)日：2022-09-01

申请号：US17187336

申请日：2021-02-26

Applicant: NetApp, Inc.

Inventor： Austino Longo ,  Christopher Cason ,  Daniel McCarthy

IPC: H04L12/927

Abstract: Systems and methods for quality of service management are provided. According to one embodiment, a non-transitory computer-readable medium comprises instructions that when executed by the processing resource cause the processing resource to receive, in a normalizing agent, one or more compute load parameters from one or more background compute processes executing on the one or more computer systems and one or more Quality of Service (QoS) parameters for one or more client compute processes executing on the one or more computer systems, convert the one or more compute load parameters to one or more normalized utilization metrics, and execute instructions that cause a processor to adjust a compute resource allocation dedicated to the one or more background compute processes based at least in part on the one or more normalized utilization metrics and the one or more QoS parameters