Object format resilient to remote object store errors

    公开(公告)号:US10360099B2

    公开(公告)日:2019-07-23

    申请号:US15581531

    申请日:2017-04-28

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.

    OBJECT FORMAT RESILIENT TO REMOTE OBJECT STORE ERRORS

    公开(公告)号:US20180314725A1

    公开(公告)日:2018-11-01

    申请号:US15581531

    申请日:2017-04-28

    Applicant: NetApp Inc.

    CPC classification number: G06F11/1004 G06F3/0619 G06F12/08 G06F16/178

    Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.

    OBJECT AND SEQUENCE NUMBER MANAGEMENT
    43.
    发明申请

    公开(公告)号:US20180314719A1

    公开(公告)日:2018-11-01

    申请号:US15581447

    申请日:2017-04-28

    Applicant: NetApp Inc.

    CPC classification number: G06F16/215 G06F16/22 G06F16/2379

    Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

    Methods and systems for efficiently storing data

    公开(公告)号:US09798496B2

    公开(公告)日:2017-10-24

    申请号:US14995001

    申请日:2016-01-13

    Applicant: NETAPP, INC.

    Abstract: Methods and systems for a networked system are provided. One method includes generating an object by a processor for storing a plurality of data chunks at a storage device, where the object includes a header segment and a data segment, the header segment providing a first offset address where an uncompressed data chunk is stored within the object and a second offset address of the object indicating a beginning of a compressed group having compressed data chunks and providing an indicator of a compression group size; reading the header segment by the processor to retrieve the second offset and the compressed group size in response to a first request for a data chunk within the compressed group; and decompressing the data chunk of the compressed group by the processor and providing the uncompressed data chunk for completing the first read request.

    Zero-copy volume move within a distributed storage system

    公开(公告)号:US12204784B1

    公开(公告)日:2025-01-21

    申请号:US18649906

    申请日:2024-04-29

    Applicant: NetApp, Inc.

    Abstract: Systems and methods for performing a zero-copy volume move between nodes of a distributed storage system are provided. In one example, an approach for performing a zero-copy volume move is proposed in which volume data may be maintained in place within a storage pod and need not be copied to move a given volume between the source node and the destination node. In one embodiment, metadata (e.g., a top-most physical volume block number (PVBN) of a node tree representing the volume at issue) of a write-anywhere file system is copied from the source node to the destination node. Since the storage pod is associated with a global PVBN space that is visible and accessible to all nodes of the distributed storage system, as a result of copying the top-mode PVBN of the volume to the destination node, anything below the top-most PVBN will automatically be visible to the destination node.

    COMBINED GARBAGE COLLECTION AND DATA INTEGRITY CHECKING FOR A DISTRIBUTED KEY-VALUE STORE

    公开(公告)号:US20240385959A1

    公开(公告)日:2024-11-21

    申请号:US18786848

    申请日:2024-07-29

    Applicant: NetApp, Inc.

    Abstract: Systems and methods are described for a streamlined garbage collection process during which data integrity checking is also performed for a distributed key-value (KV) store utilized by a distributed storage system. According to one embodiment, by making use of full or truncated block IDs (rather than an intermediate probabilistic data structure, such as a Bloom filter) for garbage collection, data integrity checking can be performed concurrently almost for free. During garbage collection, a block ID compare list may be compared to block IDs within the distributed KV store. If a particular block ID is present in the distributed KV store but is missing from the block ID compare list, the corresponding data block represents garbage to be collected. If the particular block ID is present in the block ID compare list but missing from the distributed KV store, a data integrity error has been identified.

    Combined garbage collection and data integrity checking for a distributed key-value store

    公开(公告)号:US12066933B2

    公开(公告)日:2024-08-20

    申请号:US17680484

    申请日:2022-02-25

    Applicant: NetApp, Inc.

    CPC classification number: G06F12/0253 G06F16/2246 G06F16/2272

    Abstract: Systems and methods are described for a streamlined garbage collection process during which data integrity checking is also performed for a distributed key-value (KV) store utilized by a distributed storage management system. According to one embodiment, by making use of full or truncated block IDs (rather than an intermediate probabilistic data structure, such as a Bloom filter) for garbage collection, data integrity checking can be performed concurrently almost for free. During garbage collection, a block ID compare list is compared to block IDs within the distributed KV store. If a particular block ID is present in the distributed KV store but is missing from the block ID compare list, the corresponding data block represents garbage to be collected. If the particular block ID is present in the block ID compare list but missing from the distributed KV store, a data integrity error has been identified.

Patent Agency Ranking