Automatically tuning a quality of service setting for a distributed storage system with a deep reinforcement learning agent

    公开(公告)号:US12204791B2

    公开(公告)日:2025-01-21

    申请号:US18364199

    申请日:2023-08-02

    Applicant: NetApp, Inc.

    Inventor: Tyler W. Cady

    Abstract: Systems and methods are described for using a Deep Reinforcement Learning (DRL) agent to automatically tune Quality of Service (QoS) settings of a distributed storage system (DSS). According to one embodiment, a DRL agent is trained in a simulated environment to select QoS settings (e.g., a value of one or more of a minimum IOPS parameter, a maximum IOPS parameter, and a burst IOPS parameter). The training may involve placing the DRL agent into every feasible state representing combinations of QoS settings, workload conditions, and system metrics for a period of time for multiple iterations, and rewarding the DRL agent for selecting QoS settings that minimize an objective function based on a selected measure of system load. The trained DRL agent may then be deployed to one or more DSSs to constantly update QoS settings so as to minimize the selected measure of system load.

    Zero-copy volume move within a distributed storage system

    公开(公告)号:US12204784B1

    公开(公告)日:2025-01-21

    申请号:US18649906

    申请日:2024-04-29

    Applicant: NetApp, Inc.

    Abstract: Systems and methods for performing a zero-copy volume move between nodes of a distributed storage system are provided. In one example, an approach for performing a zero-copy volume move is proposed in which volume data may be maintained in place within a storage pod and need not be copied to move a given volume between the source node and the destination node. In one embodiment, metadata (e.g., a top-most physical volume block number (PVBN) of a node tree representing the volume at issue) of a write-anywhere file system is copied from the source node to the destination node. Since the storage pod is associated with a global PVBN space that is visible and accessible to all nodes of the distributed storage system, as a result of copying the top-mode PVBN of the volume to the destination node, anything below the top-most PVBN will automatically be visible to the destination node.

    Efficient parity determination in zoned solid-state drives of a storage system

    公开(公告)号:US12204403B2

    公开(公告)日:2025-01-21

    申请号:US18377441

    申请日:2023-10-06

    Applicant: NETAPP, INC.

    Abstract: Methods and systems for a storage environment are provided. One method includes copying a data unit from a first temporary storage location corresponding to each zoned solid-state drive (ZNS SSD) of a first ZNS SSD set of a storage system to a first XOR module, while determining a first partial horizontal parity using the data unit stored in the first temporary storage location; and determining a vertical parity for each ZNS SSD of the first ZNS SSD set using the data unit provided to the first XOR module in a current cycle and vertical parity determined from a previous cycle.

    Managing Volume Snapshots in the Cloud

    公开(公告)号:US20250004986A1

    公开(公告)日:2025-01-02

    申请号:US18432243

    申请日:2024-02-05

    Applicant: NetApp, Inc.

    Abstract: Systems, methods, and machine-readable media for creating, deleting, and restoring volume snapshots in a remote data store are disclosed. A storage volume and a storage operating system are implemented in a software container. Through a user interface, a user may create a snapshot of the volume to a cloud storage. A user may also delete individual snapshots from the cloud storage. Further, deletion of a most recent snapshot may occur by awaiting deletion (though marking as deleted to the user) until a next snapshot is received. Snapshots in the cloud storage are manipulatable even after destruction of the source volume (by destruction of the container, for example). A controller outside the container is used by implementing the same API as the controller in the container had. Full restores of snapshots in the cloud are also possible even when the original container and volume have been destroyed.

    RECOVERY MECHANISM WITH SELECTIVE ORDERING AND CONCURRENT OPERATIONS

    公开(公告)号:US20250004887A1

    公开(公告)日:2025-01-02

    申请号:US18828089

    申请日:2024-09-09

    Applicant: NetApp, Inc.

    Abstract: Techniques are provided for a recovery process with selective ordering and concurrent operations in order to recover from a failure. Representations of active log structures are rebuilt within memory according to ordering values assigned to I/O operations logged within the active log structures. Representation of certain active log structures may be concurrently rebuilt based upon the active log structures comprising I/O operations that are non-overlapping within a distributed file system, have no dependencies, relate to different services, and/or target independent files. Representation of stale log structures are concurrently rebuilt within memory. While rebuilding the log structures and executing the I/O operations, a key value map is concurrently rebuilt within the memory for locating data of the I/O operations. Concurrent operations during the recovery process reduces the time to complete the recovery process, and thus reduces client downtime during the recovery process.

    LOCK RESERVATIONS FOR SHARED STORAGE

    公开(公告)号:US20250004665A1

    公开(公告)日:2025-01-02

    申请号:US18344948

    申请日:2023-06-30

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for lock reservations for shared storage. A reserve command to reserve a storage structure is received by a driver from a node. The reserve command is formatted according to a storage protocol. The driver translates the reserve command into a lease acquire command formatted according to an object store protocol and targeting an object stored within an object store and corresponding to the storage structure. A lease identifier derived from a node identifier of the node is inserted into the lease acquire command. The lease acquire command is routed to the object store for obtaining a lease on the object for granting the node exclusive write access to the object.

    Containerization and serverless thread implementation for processing objects

    公开(公告)号:US12182067B2

    公开(公告)日:2024-12-31

    申请号:US17242448

    申请日:2021-04-28

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for on-demand creation and/or utilization of containers and/or serverless threads for hosting data connector components. The data connector components can be used to perform integrity checking, anomaly detection, and file system metadata analysis associated with objects stored within an object store. The data connector components may be configured to execute machine learning functionality to perform operations and tasks. The data connector components can perform full scans or incremental scans. The data connector components may be stateless, and thus may be offlined, upgraded, onlined, and/or have tasks transferred between data connector components. Results of operations performed by the data connector components upon base objects may be stored within sibling objects.

    COMBINED GARBAGE COLLECTION AND DATA INTEGRITY CHECKING FOR A DISTRIBUTED KEY-VALUE STORE

    公开(公告)号:US20240385959A1

    公开(公告)日:2024-11-21

    申请号:US18786848

    申请日:2024-07-29

    Applicant: NetApp, Inc.

    Abstract: Systems and methods are described for a streamlined garbage collection process during which data integrity checking is also performed for a distributed key-value (KV) store utilized by a distributed storage system. According to one embodiment, by making use of full or truncated block IDs (rather than an intermediate probabilistic data structure, such as a Bloom filter) for garbage collection, data integrity checking can be performed concurrently almost for free. During garbage collection, a block ID compare list may be compared to block IDs within the distributed KV store. If a particular block ID is present in the distributed KV store but is missing from the block ID compare list, the corresponding data block represents garbage to be collected. If the particular block ID is present in the block ID compare list but missing from the distributed KV store, a data integrity error has been identified.

    ORCHESTRATOR FOR ORCHESTRATING OPERATIONS BETWEEN A COMPUTING ENVIRONMENT HOSTING VIRTUAL MACHINES AND A STORAGE ENVIRONMENT

    公开(公告)号:US20240378121A1

    公开(公告)日:2024-11-14

    申请号:US18779567

    申请日:2024-07-22

    Applicant: NetApp, Inc.

    Abstract: Techniques are provided for orchestrating operations between a storage environment and a computing environment hosting virtual machines. A virtual machine proxy, associated with a computing environment hosting a virtual machine, is accessed by an orchestrator to identify the virtual machine and properties of the virtual machine. A storage proxy, associated with a storage environment comprising a volume within which snapshots of the virtual machine are to be stored, is accessed by the orchestrator to initialize a backup procedure. The orchestrator utilizes the virtual machine proxy to create a snapshot of the virtual machine. The orchestrator utilizes the storage proxy to back up the snapshot to the volume using the backup procedure.

Patent Agency Ranking