-
公开(公告)号:US11656962B2
公开(公告)日:2023-05-23
申请号:US17657786
申请日:2022-04-04
Applicant: NetApp, Inc.
CPC classification number: G06F11/2094 , G06F11/1076 , G06F11/1088 , H03M13/154 , H03M13/373 , H03M13/3761 , G06F2201/805 , G06F2201/82
Abstract: Distributed storage systems frequently use a centralized metadata repository that stores metadata in an eventually consistent distributed database. However, a metadata repository cannot be relied upon for determining which erasure coded fragments are lost because of a storage node(s) failures. Instead, when recovering a failed storage node, a list of missing fragments is generated based on fragments stored in storage devices of available storage nodes. A storage node performing the recovery sends a request to one or more of the available storage nodes for a fragment list. The fragment list is generated, not based on a metadata database, but on scanning storage devices for fragments related to the failed storage node. The storage node performing the recovery merges retrieved lists to create a master list indicating fragments that should be regenerated for recovery of the failed storage node(s).
-
公开(公告)号:US10365848B2
公开(公告)日:2019-07-30
申请号:US15338077
申请日:2016-10-28
Applicant: NetApp, Inc.
Inventor: Tymoteusz Altman , Yi Zhang , Dheeraj Raghavender Sangamkar , Emalayan Vairavanathan
IPC: G06F3/06
Abstract: Techniques are described for reserving space on a destination node or volume for increasing the likelihood of a successful data transfer in a distributed storage environment. A reservation may be retried at one or more destinations if the reservation fails at a first destination. In some embodiments, the data-transfer process can be paused or terminated prior to data being transferred to one or more destinations if a reservation fails. Reserving space on a destination node or volume can increase the likelihood of a successful data transfer, which can increase the likelihood of efficient resources usage in a storage system.
-
3.
公开(公告)号:US20190205209A1
公开(公告)日:2019-07-04
申请号:US16210718
申请日:2018-12-05
Applicant: NetApp, Inc.
Inventor: Dheeraj Raghavender Sangamkar , Ajay Bakre , Vladimir Radu Avram , Emalayan Vairavanathan , Viswanath Chandrasekara Bharathi
CPC classification number: G06F11/1076 , G06F3/0619 , G06F3/064 , G06F3/067 , G06F3/1234 , G06F9/5038 , G06F11/1402 , G06F11/1471 , G06F11/1489 , G06F11/20 , G06F11/2069 , G06F11/2092 , G06F2201/805 , G06F2201/85 , G06F2211/1028 , H03M13/154 , H04L67/1097 , H04L69/40
Abstract: Technology is disclosed for storing data in a distributed storage system using a virtual chunk service (VCS). In the VCS based storage technique, a storage node (“node”) is split into multiple VCSs and each of the VCSs can be assigned a unique ID in the distributed storage. A set of VCSs from a set of nodes form a storage group, which also can be assigned a unique ID in the distributed storage. When a data object is received for storage, a storage group is identified for the data object, the data object is encoded to generate multiple fragments and each fragment is stored in a VCS of the identified storage group. The data recovery process is made more efficient by using metadata, e.g., VCS to storage node mapping, storage group to VCS mapping, VCS to objects mapping, which eliminates resource intensive read and write operations during recovery.
-
公开(公告)号:US20190155708A1
公开(公告)日:2019-05-23
申请号:US15820518
申请日:2017-11-22
Applicant: NetApp, Inc.
Abstract: Distributed storage systems frequently use a centralized metadata repository that stores metadata in an eventually consistent distributed database. However, a metadata repository cannot be relied upon for determining which erasure coded fragments are lost because of a storage node(s) failures. Instead, when recovering a failed storage node, a list of missing fragments is generated based on fragments stored in storage devices of available storage nodes. A storage node performing the recovery sends a request to one or more of the available storage nodes for a fragment list. The fragment list is generated, not based on a metadata database, but on scanning storage devices for fragments related to the failed storage node. The storage node performing the recovery merges retrieved lists to create a master list indicating fragments that should be regenerated for recovery of the failed storage node(s).
-
5.
公开(公告)号:US20180165155A1
公开(公告)日:2018-06-14
申请号:US15890913
申请日:2018-02-07
Applicant: NetApp, Inc.
Inventor: Dheeraj Raghavender Sangamkar , Ajay Bakre , Vladimir Radu Avram , Emalayan Vairavanathan , Viswanath Chandrasekara Bharathi
CPC classification number: G06F11/1076 , G06F3/0619 , G06F3/064 , G06F3/067 , G06F3/1234 , G06F11/1402 , G06F11/1471 , G06F11/1489 , G06F2211/1028 , H04L67/1097
Abstract: Technology is disclosed for storing data in a distributed storage system using a virtual chunk service (VCS). In the VCS based storage technique, a storage node (“node”) is split into multiple VCSs and each of the VCSs can be assigned a unique ID in the distributed storage. A set of VCSs from a set of nodes form a storage group, which also can be assigned a unique ID in the distributed storage. When a data object is received for storage, a storage group is identified for the data object, the data object is encoded to generate multiple fragments and each fragment is stored in a VCS of the identified storage group. The data recovery process is made more efficient by using metadata, e.g., VCS to storage node mapping, storage group to VCS mapping, VCS to objects mapping, which eliminates resource intensive read and write operations during recovery.
-
公开(公告)号:US09665428B2
公开(公告)日:2017-05-30
申请号:US14614856
申请日:2015-02-05
Applicant: NetApp, Inc
Inventor: Emalayan Vairavanathan , Dheeraj Sangamkar , Ajay Bakre , Vladimir Avram , Viswanath Chandrasekara Bharathi
CPC classification number: G06F11/1076 , G06F3/06 , G06F11/00 , H03M13/154 , H03M13/353 , H03M13/373 , H03M13/3761 , H04L67/108 , H04L67/1097 , H04L67/18
Abstract: A method, non-transitory computer readable medium, and storage management computing device that assists with distributing erasure coded fragments in geo-distributed storage nodes includes receiving an object and a storage reliability requirement from a client computing device. Erasure coding is performed on the received object using an erasure coding scheme. An erasure coding group from a plurality of erasure coding groups present in a plurality of geographically distributed storage nodes is determined based on the received storage reliability requirement and the erasure coding scheme. The erasure coded object is distributed to the determined erasure coding group from the plurality of erasure coding groups in the plurality of geographically distributed storage nodes.
-
公开(公告)号:US11861165B2
公开(公告)日:2024-01-02
申请号:US17646603
申请日:2021-12-30
Applicant: NetApp, Inc.
Inventor: Raymond Yu Shun Mak , Aditya Kalyanakrishnan , Song Guen Yoon , Emalayan Vairavanathan , Dheeraj Sangamkar , Chia-Chen Chu
CPC classification number: G06F3/0604 , G06F3/067 , G06F3/0644 , G06F3/0647 , G06F16/164
Abstract: A system, method, and machine-readable storage medium for analyzing a state of a data object are provided. In some embodiments, the method includes receiving, at a storage device, a metadata request for the data object from a client. The data object is composed of a plurality of segments. The method also includes selecting a subset of the plurality of segments and obtaining a segment state for each segment of the subset. Each segment state indicates whether the respective segment is accessible via a backing store. The method further includes determining a most restrictive state of the one or more segment states and sending state information to the client in response to the metadata request, the state information being derived from the most restrictive state.
-
公开(公告)号:US20220121364A1
公开(公告)日:2022-04-21
申请号:US17646603
申请日:2021-12-30
Applicant: NetApp, Inc.
Inventor: Raymond Yu Shun Mak , Aditya Kalyanakrishnan , Song Guen Yoon , Emalayan Vairavanathan , Dheeraj Sangamkar , Chia-Chen Chu
Abstract: A system, method, and machine-readable storage medium for analyzing a state of a data object are provided. In some embodiments, the method includes receiving, at a storage device, a metadata request for the data object from a client. The data object is composed of a plurality of segments. The method also includes selecting a subset of the plurality of segments and obtaining a segment state for each segment of the subset. Each segment state indicates whether the respective segment is accessible via a backing store. The method further includes determining a most restrictive state of the one or more segment states and sending state information to the client in response to the metadata request, the state information being derived from the most restrictive state.
-
公开(公告)号:US11243692B2
公开(公告)日:2022-02-08
申请号:US16359658
申请日:2019-03-20
Applicant: NetApp, Inc.
Inventor: Raymond Yu Shun Mak , Aditya Kalyanakrishnan , Song Guen Yoon , Emalayan Vairavanathan , Dheeraj Sangamkar , Chia-Chen Chu
Abstract: A system, method, and machine-readable storage medium for analyzing a state of a data object are provided. In some embodiments, the method includes receiving, at a storage device, a metadata request for the data object from a client. The data object is composed of a plurality of segments. The method also includes selecting a subset of the plurality of segments and obtaining a segment state for each segment of the subset. Each segment state indicates whether the respective segment is accessible via a backing store. The method further includes determining a most restrictive state of the one or more segment states and sending state information to the client in response to the metadata request, the state information being derived from the most restrictive state.
-
公开(公告)号:US10817393B2
公开(公告)日:2020-10-27
申请号:US16391842
申请日:2019-04-23
Applicant: NetApp, Inc.
Inventor: Dheeraj Raghavender Sangamkar , Song Guen Yoon , Emalayan Vairavanathan , Yi Zhang
IPC: G06F11/20 , G06F11/10 , H04L29/08 , G06F3/06 , G06F11/14 , G06F3/12 , H04L29/14 , H03M13/15 , G06F9/50
Abstract: To ensure that there is an elected manager among storage nodes of an erasure coding group (“ECG”), an ECG manager (“ECGM”) election process is periodically performed among available storage nodes that are configured with the software to perform the services of an ECGM. When a storage node is activated, an ECGM process of the storage node begins executing and is assigned a process identifier (“PID”). A storage node can utilize a service query framework to identify other available storage nodes and retrieve their ECGM PIDs. The storage node then selects a PID according to a criterion and elects the storage node corresponding to the selected PID to be the acting ECGM. This process is performed periodically, so even if the acting ECGM storage node fails, a new ECGM is eventually selected from the available storage nodes.
-
-
-
-
-
-
-
-
-