-
公开(公告)号:US10360099B2
公开(公告)日:2019-07-23
申请号:US15581531
申请日:2017-04-28
Applicant: NetApp Inc.
Inventor: Ananthan Subramanian , Ganga Bhavani Kondapalli , Cheryl Marie Thompson , Kevin Daniel Varghese , Anil Paul Thoppil , Qinghua Zheng
IPC: G06F11/10 , G06F12/08 , G06F3/06 , G06F16/178
Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.
-
公开(公告)号:US20180314725A1
公开(公告)日:2018-11-01
申请号:US15581531
申请日:2017-04-28
Applicant: NetApp Inc.
Inventor: Ananthan Subramanian , Ganga Bhavani Kondapalli , Cheryl Marie Thompson , Kevin Daniel Varghese , Anil Paul Thoppil , Qinghua Zheng
CPC classification number: G06F11/1004 , G06F3/0619 , G06F12/08 , G06F16/178
Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.
-
公开(公告)号:US20180314719A1
公开(公告)日:2018-11-01
申请号:US15581447
申请日:2017-04-28
Applicant: NetApp Inc.
Inventor: Ananthan Subramanian , Sridevi Jantli , Anil Paul Thoppil , Cheryl Marie Thompson , Qinghua Zheng
IPC: G06F17/30
CPC classification number: G06F16/215 , G06F16/22 , G06F16/2379
Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.
-
公开(公告)号:US09798496B2
公开(公告)日:2017-10-24
申请号:US14995001
申请日:2016-01-13
Applicant: NETAPP, INC.
Inventor: Kevin Daniel Varghese , Anil Paul Thoppil
CPC classification number: G06F3/0661 , G06F3/0608 , G06F3/0611 , G06F3/0659 , G06F3/067 , G06F3/0688 , G06F3/0689 , H04L67/1097
Abstract: Methods and systems for a networked system are provided. One method includes generating an object by a processor for storing a plurality of data chunks at a storage device, where the object includes a header segment and a data segment, the header segment providing a first offset address where an uncompressed data chunk is stored within the object and a second offset address of the object indicating a beginning of a compressed group having compressed data chunks and providing an indicator of a compression group size; reading the header segment by the processor to retrieve the second offset and the compressed group size in response to a first request for a data chunk within the compressed group; and decompressing the data chunk of the compressed group by the processor and providing the uncompressed data chunk for completing the first read request.
-
公开(公告)号:US09792043B2
公开(公告)日:2017-10-17
申请号:US14994924
申请日:2016-01-13
Applicant: NETAPP, INC.
CPC classification number: G06F3/0608 , G06F3/0619 , G06F3/0647 , G06F3/065 , G06F3/0652 , G06F3/0665 , G06F3/0667 , G06F3/067 , G06F3/0685 , G06F12/0253 , G06F12/0261
Abstract: Methods and systems for a networked storage environment are provided. One method includes scanning a first data structure by a processor executing instructions out of a memory for a storage operating system to determine whether any data chunk of a first object stored at a first storage tier is referenced by the storage operating; when the storage operating system references a certain number of data chunks, the processor using an object staging data structure to identify a second object that is in the process of being built with space for transferring the certain number of data chunks from the first object to the second object; and updating information regarding the second object at a transfer log with location information of the certain number of data chunks at the first storage tier.
-
公开(公告)号:US20170199675A1
公开(公告)日:2017-07-13
申请号:US14994924
申请日:2016-01-13
Applicant: NETAPP, INC.
CPC classification number: G06F3/0608 , G06F3/0619 , G06F3/0647 , G06F3/065 , G06F3/0652 , G06F3/0665 , G06F3/0667 , G06F3/067 , G06F3/0685 , G06F12/0253 , G06F12/0261
Abstract: Methods and systems for a networked storage environment are provided. One method includes scanning a first data structure by a processor executing instructions out of a memory for a storage operating system to determine whether any data chunk of a first object stored at a first storage tier is referenced by the storage operating; when the storage operating system references a certain number of data chunks, the processor using an object staging data structure to identify a second object that is in the process of being built with space for transferring the certain number of data chunks from the first object to the second object; and updating information regarding the second object at a transfer log with location information of the certain number of data chunks at the first storage tier.
-
公开(公告)号:US12204784B1
公开(公告)日:2025-01-21
申请号:US18649906
申请日:2024-04-29
Applicant: NetApp, Inc.
Inventor: Anil Paul Thoppil , Ananthan Subramanian , Kevin Daniel Varghese , Garima Choudhary , Qinghua Zheng
Abstract: Systems and methods for performing a zero-copy volume move between nodes of a distributed storage system are provided. In one example, an approach for performing a zero-copy volume move is proposed in which volume data may be maintained in place within a storage pod and need not be copied to move a given volume between the source node and the destination node. In one embodiment, metadata (e.g., a top-most physical volume block number (PVBN) of a node tree representing the volume at issue) of a write-anywhere file system is copied from the source node to the destination node. Since the storage pod is associated with a global PVBN space that is visible and accessible to all nodes of the distributed storage system, as a result of copying the top-mode PVBN of the volume to the destination node, anything below the top-most PVBN will automatically be visible to the destination node.
-
48.
公开(公告)号:US20240385959A1
公开(公告)日:2024-11-21
申请号:US18786848
申请日:2024-07-29
Applicant: NetApp, Inc.
Inventor: Wei Sun , Mark David Olson , Anil Paul Thoppil
Abstract: Systems and methods are described for a streamlined garbage collection process during which data integrity checking is also performed for a distributed key-value (KV) store utilized by a distributed storage system. According to one embodiment, by making use of full or truncated block IDs (rather than an intermediate probabilistic data structure, such as a Bloom filter) for garbage collection, data integrity checking can be performed concurrently almost for free. During garbage collection, a block ID compare list may be compared to block IDs within the distributed KV store. If a particular block ID is present in the distributed KV store but is missing from the block ID compare list, the corresponding data block represents garbage to be collected. If the particular block ID is present in the block ID compare list but missing from the distributed KV store, a data integrity error has been identified.
-
49.
公开(公告)号:US12066933B2
公开(公告)日:2024-08-20
申请号:US17680484
申请日:2022-02-25
Applicant: NetApp, Inc.
Inventor: Wei Sun , Mark David Olson , Anil Paul Thoppil
CPC classification number: G06F12/0253 , G06F16/2246 , G06F16/2272
Abstract: Systems and methods are described for a streamlined garbage collection process during which data integrity checking is also performed for a distributed key-value (KV) store utilized by a distributed storage management system. According to one embodiment, by making use of full or truncated block IDs (rather than an intermediate probabilistic data structure, such as a Bloom filter) for garbage collection, data integrity checking can be performed concurrently almost for free. During garbage collection, a block ID compare list is compared to block IDs within the distributed KV store. If a particular block ID is present in the distributed KV store but is missing from the block ID compare list, the corresponding data block represents garbage to be collected. If the particular block ID is present in the block ID compare list but missing from the distributed KV store, a data integrity error has been identified.
-
公开(公告)号:US11983080B2
公开(公告)日:2024-05-14
申请号:US17680621
申请日:2022-02-25
Applicant: NetApp, Inc.
Inventor: Wei Sun , Anil Paul Thoppil , Anne Maria Vasu
CPC classification number: G06F11/1662 , G06F3/0622 , G06F3/064 , G06F3/0679 , G06F11/1088 , G06F11/3034 , G06F16/27
Abstract: Systems and methods that make use of cluster-level redundancy within a distributed storage management system to address various node-level error scenarios are provided. Rather than making use of a generalized one-size-fits-all approach in an effort to reduce complexity, an approach tailored to the node-level error scenario at issue may be performed to avoid doing more than necessary. According to one embodiment, responsive to identification of a failed RAID stripe by a node of a cluster of a distributed storage management system, for each block ID of multiple block IDs associated with the failed RAID stripe, a data block is restored corresponding to the block ID by reading the data block from another node of the cluster having a redundant copy of the data block; and writing the redundant copy of the data block to a storage area of the node that is unaffected by the failed RAID stripe.
-
-
-
-
-
-
-
-
-