IN-MEMORY HASH ENTRIES AND HASHES USED TO IMPROVE KEY SEARCH OPERATIONS FOR KEYS OF A KEY VALUE STORE

    公开(公告)号:US20230350810A1

    公开(公告)日:2023-11-02

    申请号:US17732098

    申请日:2022-04-28

    Applicant: NetApp Inc.

    CPC classification number: G06F12/1018

    Abstract: Techniques are provided for implementing a hash building process and an append hash building process. The hash building process builds in-memory hash entries for bins of keys stored within sorted logs of a log structured merge tree used to store keys of a key-value store. The in-memory hash entries can be used to identify the starting locations of bins of keys within the log structured merge tree so that a key within a bin can be searched for from the starting location of the bin as opposed to having to search the entire log structured merge tree. The append hash building process builds two hashes that can be used to more efficiently locate keys and/or ranges of keys within an unsorted append log that would otherwise require a time consuming binary search of the entire append log.

    Techniques for asynchronous snapshot invalidation

    公开(公告)号:US10853328B2

    公开(公告)日:2020-12-01

    申请号:US16015730

    申请日:2018-06-22

    Applicant: NetApp Inc.

    Abstract: Asynchronous snapshot invalidation techniques are described. According to various such techniques, an enhanced file handle structure may be defined that includes a snapshot generation ID that is to comprise a value that singularly identifies a snapshot performed at a particular point in time. In some embodiments, when a snapshot ID assigned to that snapshot is reused at a subsequent point in time, a different snapshot generation ID may be assigned to that subsequent snapshot. With respect to an in-core cache, the differing snapshot generation IDs may eliminate unacceptable ambiguity regarding respective file information sets corresponding to the initial and subsequent snapshots sharing the same snapshot ID. As a result, obsolete file information sets may be cleared from the in-core cache asynchronously, enabling improved performance. The embodiments are not limited in this context.

    Defragmentation for log structured merge tree to improve read and write amplification

    公开(公告)号:US11971859B2

    公开(公告)日:2024-04-30

    申请号:US17732046

    申请日:2022-04-28

    Applicant: NetApp Inc.

    CPC classification number: G06F16/1748 G06F16/182

    Abstract: Techniques are provided for implementing a defragmentation process during a merge operation performed by a re-compaction process upon a log structured merge tree. The log structured merge tree is used to store keys of key-value pairs within a key-value store. As the log structured merge tree fills with keys over time, the re-compaction process is performed to merge keys down to lower levels of the log structured merge tree to re-compact the keys. Re-compaction can result in fragmentation because there is a lack of spatial locality of where the re-compaction operations re-writes the keys within storage. Fragmentation increases read and write amplification when accessing the keys stored in different locations within the storage. Accordingly, the defragmentation process is performed during a last merge operation of the re-compaction process in order to store keys together within the storage, thus reducing read and write amplification when accessing the keys.

    Network storage failover systems and associated methods

    公开(公告)号:US11216350B2

    公开(公告)日:2022-01-04

    申请号:US16855853

    申请日:2020-04-22

    Applicant: NETAPP, INC.

    Abstract: Failover methods and systems for a networked storage environment are provided. A metadata data structure is generated, before starting a replay of entries at a log stored in a non-volatile memory of a second storage node, during a failover operation initiated in response to a failure at a first storage node. The second storage node operates as a partner node of the first storage node, and the metadata structure stores a metadata attribute of each log entry. Furthermore, the metadata attribute of each log entry is persistently stored. The persistently stored metadata attribute is used to respond to a read request received during the replay by the second storage node, while a write request metadata attribute of a write request is used for executing the write request received by the second storage node during the replay.

    Metadata compaction
    5.
    发明授权

    公开(公告)号:US10852994B2

    公开(公告)日:2020-12-01

    申请号:US16365956

    申请日:2019-03-27

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object. Storing compacted indirect blocks into memory allows for more block numbers to be stored within memory.

    System and method for invalidation walk-through of inodes
    6.
    发明授权
    System and method for invalidation walk-through of inodes 有权
    inode无效传播的系统和方法

    公开(公告)号:US09348532B1

    公开(公告)日:2016-05-24

    申请号:US13776348

    申请日:2013-02-25

    Applicant: NetApp, Inc.

    Abstract: A system and method for deferring an invalidation walk-through of a volume in memory and a later invalidation walk-through of inodes the volume are described herein. Node A may store volume information for one or more volumes to memory including inodes of the volume storing data for the volume. Node A performs a primary procedure that may include removing volumes from the memory, such as a procedure that transfers ownership of the volumes to node B. Removing a volume from memory comprises removing its volume information from memory. Node A performs an invalidation deferment procedure that marks each volume as stale for preventing data of the volumes to be served and sends a completion message to node B which begins serving data of the volume. After node B begins serving data of the volume, node A performs an invalidation walk-through procedure to remove inodes of the volume from memory.

    Abstract translation: 本文描述了用于推迟存储器中的卷的无效化步骤的系统和方法,以及后来的inode的无效化步骤。 节点A可以将一个或多个卷的卷信息存储在存储器的存储器中,该存储器包括存储该卷的数据的卷的inode。 节点A执行主要过程,其中可能包括从内存中删除卷,例如将卷的所有权转移到节点B的过程。从内存中删除卷包括从内存中删除其卷信息。 节点A执行无效延迟过程,该过程将每个卷标记为过期,以防止要服务的卷的数据,并将完成消息发送到开始提供卷的数据的节点B. 在节点B开始提供卷的数据之后,节点A执行无效化步骤以从存储器中删除卷的inode。

    Metadata compaction
    8.
    发明授权

    公开(公告)号:US11467774B2

    公开(公告)日:2022-10-11

    申请号:US17083945

    申请日:2020-10-29

    Applicant: NetApp Inc.

    Abstract: Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object.

    Network Storage Failover Systems and Associated Methods

    公开(公告)号:US20220283915A1

    公开(公告)日:2022-09-08

    申请号:US17751944

    申请日:2022-05-24

    Applicant: NETAPP, INC.

    Abstract: Failover methods and systems for a networked storage environment are provided. In one aspect, a read request associated with a first storage object is received, during a replay of entries of a log stored in a non-volatile memory of a second storage node for a failover operation initiated in response to a failure at a first storage node. The second storage node operates as a partner node of the first storage node. The read request is processed using a filtering data structure that is generated from the log prior to the replay and identifies each log entry. The read request is processed when the log does not have an entry associated with the read request, and when the filtering data structure includes an entry associated with the read request, the requested data is located at the non-volatile memory.

Patent Agency Ranking