DEFRAGMENTATION FOR LOG STRUCTURED MERGE TREE TO IMPROVE READ AND WRITE AMPLIFICATION

    公开(公告)号:US20230350850A1

    公开(公告)日:2023-11-02

    申请号:US17732046

    申请日:2022-04-28

    Applicant: NetApp Inc.

    CPC classification number: G06F3/0605 G06F3/0685 G06F3/0649

    Abstract: Techniques are provided for implementing a defragmentation process during a merge operation performed by a re-compaction process upon a log structured merge tree. The log structured merge tree is used to store keys of key-value pairs within a key-value store. As the log structured merge tree fills with keys over time, the re-compaction process is performed to merge keys down to lower levels of the log structured merge tree to re-compact the keys. Re-compaction can result in fragmentation because there is a lack of spatial locality of where the re-compaction operations re-writes the keys within storage. Fragmentation increases read and write amplification when accessing the keys stored in different locations within the storage. Accordingly, the defragmentation process is performed during a last merge operation of the re-compaction process in order to store keys together within the storage, thus reducing read and write amplification when accessing the keys.

    PREFETCHING KEYS FOR GARBAGE COLLECTION
    24.
    发明公开

    公开(公告)号:US20230350610A1

    公开(公告)日:2023-11-02

    申请号:US17732065

    申请日:2022-04-28

    Applicant: NetApp Inc.

    CPC classification number: G06F3/0652 G06F3/0602 G06F3/068

    Abstract: Techniques are provided for implementing a garbage collection process and a prediction read ahead mechanism to prefetch keys into memory to improve the efficiency and speed of the garbage collection process. A log structured merge tree is used to store keys of key-value pairs within a key-value store. If a key is no longer referenced by any worker nodes of a distributed storage architecture, then the key can be freed to store other data. Accordingly, garbage collection is performed to identify and free unused keys. The speed and efficiency of garbage collection is improved by dynamically adjusting the amount and rate at which keys are prefetched from disk and cached into faster memory for processing by the garbage collection process.

    OBJECT FORMAT RESILIENT TO REMOTE OBJECT STORE ERRORS

    公开(公告)号:US20220147418A1

    公开(公告)日:2022-05-12

    申请号:US17582221

    申请日:2022-01-24

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.

    Object format resilient to remote object store errors

    公开(公告)号:US11231989B2

    公开(公告)日:2022-01-25

    申请号:US17100905

    申请日:2020-11-22

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.

    OBJECT STORE MIRRORING
    30.
    发明申请

    公开(公告)号:US20200327141A1

    公开(公告)日:2020-10-15

    申请号:US16389019

    申请日:2019-04-19

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for object store mirroring. Data within a storage tier of a node may be determined as being data to tier out to a primary object store based upon a property of the data. A first object is generated to comprise the data. A second object is generated to comprise the data. The first object is transmitted to the primary data store for storage in parallel with the second object being transmitted to a mirror object store for storage. Tiering of the data is designated as successful once acknowledgements are received from both the primary object that the first object was stored and the mirror object store that the second object was stored.

Patent Agency Ranking