公开(公告)号：US09830103B2

公开(公告)日：2017-11-28

申请号：US14988435

申请日：2016-01-05

Applicant: NetApp, Inc.

Inventor： Anshul Pundir ,  Ling Zheng ,  David Brittain Bolen

IPC: G06F3/06

CPC classification number: G06F3/0644 ,  G06F3/0608 ,  G06F3/067

Abstract: A technique enables recovery of storage space trapped in an extent store from overlapping write requests associated with metadata describing volume logical storage addresses for data in the extent store. The metadata is organized as metadata entries in a multi-level dense tree metadata structure. When a level of the dense tree is full, the metadata entries of the level are merged with a next lower level of the dense tree in accordance with a dense tree merge operation. The technique may be invoked during the merge operation to process the metadata entries associated with the overlapping write requests involved in the merge operation. Processing of the overlapping write requests during the merge operation may partially overwrite extents which, in turn, may result in logical storage space being trapped in the extent store. The technique may perform read-modify-write (RMW) operations on the partially overwritten extents to recapture that trapped space.

公开(公告)号：US20170132243A1

公开(公告)日：2017-05-11

申请号：US15410789

申请日：2017-01-20

Applicant: NetApp, Inc.

Inventor： Blake Lewis ,  Ling Zheng ,  Craig Johnston ,  Vinod Daga

IPC: G06F17/30 ,  G06F3/06

CPC classification number: G06F17/30156 ,  G06F3/0608 ,  G06F3/0641 ,  G06F3/0659 ,  G06F3/067

Abstract: A system can maintain multiple queues for deduplication requests of different priorities. The system can also designate priority of storage units. The scheduling priority of a deduplication request is based on the priority of the storage unit indicated in the deduplication request and a trigger for the deduplication request.

公开(公告)号：US20160077744A1

公开(公告)日：2016-03-17

申请号：US14484061

申请日：2014-09-11

Applicant: NETAPP, INC.

Inventor： Anshul Pundir ,  Ashwin Pednekar ,  Srinath Krishnamachari ,  Ling Zheng

IPC: G06F3/06

CPC classification number: G06F3/0608 ,  G06F3/0611 ,  G06F3/0652 ,  G06F3/0655 ,  G06F3/0665 ,  G06F3/067 ,  G06F3/0688

Abstract: A deferred refcount update technique efficiently frees storage space for metadata (associated with data) to be deleted during a merge operation managed by a volume layer of a node. The metadata is illustratively volume metadata embodied as mappings from logical block addresses (LBAs) of a logical unit (LUN) to extent keys maintained by an extent store layer of the node. One or more requests to delete (or overwrite) an LBA range within a LUN may be captured as page keys associated with metadata pages during the merge operation and the storage space associated with those metadata pages may be freed in an out-of-band fashion. The page keys of the metadata pages may be persistently recorded in a reference count (refcount) log to thereby allow the merge operation to complete without resolving deletion of the keys. A batch of page keys may be organized as one or more delete requests and, once the merge completes, the keys may be inserted into the refcount log. Subsequently, a deferred reference count update process may be spawned (instantiated) to walk through the page keys stored in the refcount log and delete each key, e.g., from the extent store layer, independently and out-of-band from the merge operation.

Abstract translation: 延迟重新计费更新技术有效地释放了在由节点的卷层管理的合并操作期间要删除的元数据（与数据相关联）的存储空间。 元数据示例性地是体现为从逻辑单元（LUN）的逻辑块地址（LBA）到由节点的扩展区存储层维护的扩展密钥的映射的卷元数据。 删除（或覆盖）LUN中的LBA范围的一个或多个请求可以被捕获为在合并操作期间与元数据页相关联的页面键，并且与那些元数据页相关联的存储空间可以以带外方式释放 。 元数据页面的页面键可以被持久地记录在引用计数（引用计数）日志中，从而允许合并操作完成而不解决键的删除。 一批页面键可以被组织为一个或多个删除请求，并且一旦合并完成，则可以将密钥插入到引用计数日志中。 随后，可以产生（实例化）延迟引用计数更新处理以遍历存储在引用计数日志中的页面密钥，并且例如从扩展存储层中删除每个密钥，从合并操作中独立地进行带外删除。

公开(公告)号：US09286006B2

公开(公告)日：2016-03-15

申请号：US13857835

申请日：2013-04-05

Applicant: NetApp, Inc.

Inventor： Blake Lewis ,  Ling Zheng ,  Craig Johnston ,  Vinod Daga

IPC: G06F17/00 ,  G06F3/06

CPC classification number: G06F17/30156 ,  G06F3/0608 ,  G06F3/0641 ,  G06F3/0659 ,  G06F3/067

Abstract: Systems for deduplicating one or more storage units of a storage system provide a scheduler, which is operable to select at least one storage unit (e.g. a storage volume) for deduplication and perform a deduplication process, which removes duplicate data blocks from the selected storage volume. The systems are operable to determine the state of one or more storage units and manage deduplication requests in part based state information. The system is further operable to manage user generated requests and manage deduplication requests in part based on user input information. The system may include a rules engine which prioritizes system operations including determining an order in which to perform state-gathering information and determining an order in which to perform deduplication. The system is further operable to determine the order in which storage units are processed.

Abstract translation: 用于对存储系统的一个或多个存储单元进行重复数据删除的系统提供调度器，其可操作以选择用于重复数据删除的至少一个存储单元（例如，存储卷），并执行重复数据删除过程，其从所选存储卷中去除重复的数据块 。 该系统可操作以确定一个或多个存储单元的状态，并在部分状态信息中管理重复数据删除请求。 该系统进一步可操作以部分地基于用户输入信息来管理用户生成的请求并管理重复数据删除请求。 系统可以包括规则引擎，其优先化系统操作，包括确定执行状态收集信息的顺序以及确定执行重复数据删除的顺序。 该系统还可操作以确定处理存储单元的顺序。

公开(公告)号：US20150370485A1

公开(公告)日：2015-12-24

申请号：US14838614

申请日：2015-08-28

Applicant: NetApp, Inc.

Inventor： Ling Zheng ,  Blake H. Lewis ,  Kayuri H. Patel

IPC: G06F3/06 ,  G06F11/14

CPC classification number: G06F3/065 ,  G06F3/0604 ,  G06F3/0619 ,  G06F3/0659 ,  G06F3/067 ,  G06F3/0685 ,  G06F3/0688 ,  G06F3/0689 ,  G06F11/1464 ,  G06F17/30575 ,  G06F2201/84 ,  G06F2212/262

Abstract: In one embodiment, a node coupled to one or more storage devices executes a storage input/output (I/O) stack having a volume layer, a persistence layer and an administration layer that interact to create a copy of a parent volume associated with a storage container on the one or more storage devices. A copy create start message is received at the persistence layer from the administration layer. The persistence layer ensures that dirty data for the parent volume is incorporated into the copy of the parent volume. New data for the parent volume received at the persistence layer during creation of the copy of the parent volume is prevented from incorporation into the copy of the parent volume. A reply to the copy create start message is sent from the persistence layer to the administration layer to initiate the creation of the copy of the parent volume at the volume layer.

Abstract translation: 在一个实施例中，耦合到一个或多个存储设备的节点执行具有卷层，持久层和管理层的存储输入/输出（I / O）堆栈，所述卷层，持久层和管理层进行交互以创建与 存储容器在一个或多个存储设备上。 在管理层的持久层收到复制创建开始消息。 持久层确保父卷的脏数据合并到父卷的副本中。 在创建父卷的副本期间，在持久层处收到的父卷的新数据将被阻止并入父卷的副本。 将复制创建开始消息的答复从持久层发送到管理层，以启动在卷层创建父卷的副本。

公开(公告)号：US20150134879A1

公开(公告)日：2015-05-14

申请号：US14078146

申请日：2013-11-12

Applicant: NetApp, Inc.

Inventor： Ling Zheng ,  Blake H. Lewis ,  Kayuri H. Patel

IPC: G06F3/06

CPC classification number: G06F3/065 ,  G06F3/0604 ,  G06F3/0619 ,  G06F3/0659 ,  G06F3/067 ,  G06F3/0685 ,  G06F3/0688 ,  G06F3/0689 ,  G06F11/1464 ,  G06F17/30575 ,  G06F2201/84 ,  G06F2212/262

Abstract: In one embodiment, a node coupled to one or more storage devices executes a storage input/output (I/O) stack having a volume layer that manages volume metadata. The volume metadata is organized as one or more dense tree metadata structures having a top level residing in memory and lower levels residing on the one or more storage devices. The dense tree metadata structures include a first dense tree metadata structure associated with a parent volume and a second dense tree metadata structure associated with a copy of the parent volume. The top level of the first dense tree metadata structure may be copied to the second dense tree metadata structure. The lower levels of the first dense tree metadata structure are initially shared with the second dense tree metadata structure. The shared lower levels may eventually be split as the parent volume diverges from the copy of the parent volume.

Abstract translation: 在一个实施例中，耦合到一个或多个存储设备的节点执行具有管理卷元数据的卷层的存储输入/输出（I / O）堆栈。 卷元数据被组织为一个或多个密集树元数据结构，其具有驻留在存储器中的顶级和驻留在一个或多个存储设备上的较低级别。 密集树元数据结构包括与父卷相关联的第一密集树元数据结构和与父卷的副本相关联的第二密集树元数据结构。 第一密集树元数据结构的顶层可以被复制到第二密集树元数据结构。 第一密集树元数据结构的较低级别最初与第二密集树元数据结构共享。 共享的较低级别最终可能会因父体卷从父卷的副本分歧而被拆分。

公开(公告)号：US20150134616A1

公开(公告)日：2015-05-14

申请号：US14162106

申请日：2014-01-23

Applicant: NetApp, Inc.

Inventor： Ling Zheng ,  Blake H. Lewis ,  Kayuri H. Patel

IPC: G06F17/30

CPC classification number: G06F3/065 ,  G06F3/0604 ,  G06F3/0619 ,  G06F3/0659 ,  G06F3/067 ,  G06F3/0685 ,  G06F3/0688 ,  G06F3/0689 ,  G06F11/1464 ,  G06F17/30575 ,  G06F2201/84 ,  G06F2212/262

Abstract: In one embodiment, snapshots and/or clones of storage objects are created and managed by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. Illustratively, the snapshots and clones may be represented as independent volumes, and embodied as respective read-only copies (snapshots) and read-write copies (clones) of a parent volume. Volume metadata is illustratively organized as one or more multi-level dense tree metadata structures, wherein each level of the dense tree metadata structure (dense tree) includes volume metadata entries for storing the metadata. Each snapshot/clone may be derived from a dense tree of the parent volume (parent dense tree). Portions of the parent dense tree may be shared with the snapshot/clone.

Abstract translation: 在一个实施例中，由集群的一个或多个节点上执行的存储输入/输出（I / O）栈的卷层创建和管理存储对象的快照和/或克隆。 说明性地，快照和克隆可以表示为独立卷，并且被体现为父卷的相应的只读副本（快照）和读写副本（克隆）。 卷元数据被示例性地组织为一个或多个多级密集树元数据结构，其中密集树元数据结构（密集树）的每个级别包括用于存储元数据的卷元数据条目。 每个快照/克隆可以从父卷（父密集树）的密集树导出。 父密度树的部分可能与快照/克隆共享。

公开(公告)号：US20140358873A1

公开(公告)日：2014-12-04

申请号：US14460032

申请日：2014-08-14

Applicant: NETAPP, INC.

Inventor： Vinod Kumar Daga ,  Craig Anthony Johnston ,  Ling Zheng

IPC: G06F17/30

CPC classification number: G06F17/30303 ,  G06F17/30079 ,  G06F17/3015 ,  G06F17/30156

Abstract: A method performed in a system that has a plurality of volumes stored to storage hardware, the method including generating, for each of the volumes, a respective space saving potential iteratively over time and scheduling space saving operations among the plurality of volumes by analyzing each of the volumes for space saving potential and assigning priority of resources based at least in part on space saving potential.

Abstract translation: 一种在具有存储到存储硬件的多个卷的系统中执行的方法，所述方法包括针对每个卷生成随时间迭代的相应的空间节省潜力，并且通过分析每个卷中的每个 节省空间的潜力，并至少部分依赖节省空间的潜力来分配资源的优先级。

公开(公告)号：US08892818B1

公开(公告)日：2014-11-18

申请号：US14162228

申请日：2014-01-23

Applicant: NetApp, Inc.

Inventor： Ling Zheng ,  Blake H. Lewis ,  Kayuri H. Patel

IPC: G06F12/00 ,  G06F17/30

CPC classification number: G06F17/30327 ,  G06F3/0611 ,  G06F3/0619 ,  G06F3/0638 ,  G06F3/0641 ,  G06F3/0655 ,  G06F3/0665 ,  G06F3/067 ,  G06F3/0689 ,  G06F17/30 ,  G06F17/30292

Abstract: The embodiments described herein are directed to an organization of metadata managed by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. The metadata managed by the volume layer, i.e., the volume metadata, is illustratively embodied as mappings from addresses, i.e., logical block addresses (LBAs), of a logical unit (LUN) accessible by a host to durable extent keys maintained by an extent store layer of the storage I/O stack. In an embodiment, the volume layer organizes the volume metadata as a mapping data structure, i.e., a dense tree metadata structure, which represents successive points in time to enable efficient access to the metadata.

Abstract translation: 这里描述的实施例涉及由在集群的一个或多个节点上执行的存储输入/输出（I / O）栈的卷层管理的元数据的组织。 由卷层管理的元数据（即卷元数据）被示意性地体现为从主机可访问的逻辑单元（LUN）的地址（即，逻辑块地址（LBA））到可由扩展区维护的持久范围密钥的映射 存储I / O堆栈的存储层。 在一个实施例中，卷层将卷元数据组织为映射数据结构，即密集树元数据结构，其表示连续的时间点以实现对元数据的有效访问。

公开(公告)号：US20140304239A1

公开(公告)日：2014-10-09

申请号：US13857835

申请日：2013-04-05

Applicant: NETAPP, INC.

Inventor： Blake Lewis ,  Ling Zheng ,  Craig Johnston ,  Vinod Daga

IPC: G06F17/30

CPC classification number: G06F17/30156 ,  G06F3/0608 ,  G06F3/0641 ,  G06F3/0659 ,  G06F3/067

Abstract: Systems for deduplicating one or more storage units of a storage system provide a scheduler, which is operable to select at least one storage unit (e.g. a storage volume) for deduplication and perform a deduplication process, which removes duplicate data blocks from the selected storage volume. The systems are operable to determine the state of one or more storage units and manage deduplication requests in part based state information. The system is further operable to manage user generated requests and manage deduplication requests in part based on user input information. The system may include a rules engine which prioritizes system operations including determining an order in which to perform state-gathering information and determining an order in which to perform deduplication. The system is further operable to determine the order in which storage units are processed.

Abstract translation: 用于对存储系统的一个或多个存储单元进行重复数据删除的系统提供调度器，其可操作以选择用于重复数据删除的至少一个存储单元（例如，存储卷），并执行重复数据删除过程，其从所选存储卷中去除重复的数据块 。 该系统可操作以确定一个或多个存储单元的状态，并在部分状态信息中管理重复数据删除请求。 该系统进一步可操作以部分地基于用户输入信息来管理用户生成的请求并管理重复数据删除请求。 系统可以包括规则引擎，其优先化系统操作，包括确定执行状态收集信息的顺序以及确定执行重复数据删除的顺序。 该系统还可操作以确定处理存储单元的顺序。