公开(公告)号：US09405473B2

公开(公告)日：2016-08-02

申请号：US14927607

申请日：2015-10-30

Applicant: NetApp, Inc.

Inventor： Ling Zheng ,  Blake H. Lewis

IPC: G06F3/06 ,  G06F17/30

CPC classification number: G06F3/061 ,  G06F3/0604 ,  G06F3/0644 ,  G06F3/0646 ,  G06F3/0685 ,  G06F3/0688 ,  G06F17/30327

Abstract: The embodiments described herein are directed to efficient merging of metadata managed by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. The metadata managed by the volume layer, i.e., the volume metadata, is illustratively organized as a multi-level dense tree metadata structure, wherein each level of the dense tree metadata structure (dense tree) includes volume metadata entries for storing the volume metadata. The volume metadata entries of an upper level of the dense tree metadata structure are merged with the volume metadata entries of a next lower level of the dense tree metadata structure when the upper level is full. The volume metadata entries of the merged levels are organized as metadata pages and stored as one or more files on the SSDs.

Abstract translation: 这里描述的实施例涉及由在集群的一个或多个节点上执行的存储输入/输出（I / O）堆栈的卷层管理的元数据的有效合并。 由卷层管理的元数据，即卷元数据被说明性地组织为多级密集树元数据结构，其中密集树元数据结构（密集树）的每个级别包括用于存储卷元数据的卷元数据条目。 密集树元数据结构的较高级别的卷元数据条目与较高级别已满的密集树元数据结构的下一级较低级别的卷元数据条目合并。 合并级别的卷元数据条目被组织为元数据页面，并作为一个或多个文件存储在SSD上。

公开(公告)号：US08880788B1

公开(公告)日：2014-11-04

申请号：US14160991

申请日：2014-01-22

Applicant: NetApp, Inc.

Inventor： Rajesh Sundaram ,  Stephen Daniel ,  Jeffrey S. Kimmel ,  Blake H. Lewis

IPC: G06F12/00 ,  G06F3/06

CPC classification number: G06F17/30138 ,  G06F3/0604 ,  G06F3/0608 ,  G06F3/0616 ,  G06F3/064 ,  G06F3/0643 ,  G06F3/0644 ,  G06F3/0647 ,  G06F3/0659 ,  G06F3/0679 ,  G06F3/0685 ,  G06F3/0688 ,  G06F12/0246 ,  G06F17/30185 ,  G06F17/30218 ,  G06F2003/0697 ,  G06F2212/2022 ,  G06F2212/7208

Abstract: In one embodiment, a flash-optimized, log-structured layer of a file system of a storage input/output (I/O) stack executes on one or more nodes of a cluster. The log-structured layer of the file system provides sequential storage of data and metadata on solid state drives (SSDs) to reduce write amplification, while leveraging variable compression and variable length data features of the storage I/O stack. The data may be organized as an arbitrary number of variable-length extents of one or more host-visible logical units (LUNs). The metadata may include mappings from host-visible logical block address ranges of a LUN to extent keys, as well as mappings of the extent keys to SSD storage locations of the extents. The storage location of an extent on SSD is effectively “virtualized” by its mapped extent key such that relocation of the extent on SSD does not require update to volume layer metadata.

Abstract translation: 在一个实施例中，存储输入/输出（I / O）堆栈的文件系统的闪存优化的日志结构化层在集群的一个或多个节点上执行。 文件系统的日志结构化层在固态驱动器（SSD）上提供数据和元数据的顺序存储，以减少写入放大，同时利用存储I / O堆栈的可变压缩和可变长度数据特征。 数据可以被组织为一个或多个主机可见逻辑单元（LUN）的任意数量的可变长度盘区。 元数据可以包括从LUN到扩展密钥的主机可见逻辑块地址范围的映射，以及扩展密钥到扩展区的SSD存储位置的映射。 SSD上的盘区的存储位置被其映射的盘区密钥有效地“虚拟化”，使得SSD上盘区的重新定位不需要更新到卷层元数据。

公开(公告)号：US20140082281A1

公开(公告)日：2014-03-20

申请号：US14077812

申请日：2013-11-12

Applicant: NetApp, Inc.

Inventor： John K. Edwards ,  Blake H. Lewis ,  Robert M. English ,  Eric Hamilton ,  Peter F. Corbett

IPC: G06F3/06

CPC classification number: G06F3/0689 ,  G06F3/0619 ,  G06F3/0665 ,  G06F17/30233

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols. The file system layout extends the file system layout of a conventional write anywhere file layout system implementation, yet maintains performance properties of the conventional implementation.

Abstract translation: 文件系统布局将底层物理卷分配到存储系统的一个或多个虚拟卷（vvols）中。 底层物理卷是包含存储系统的一组或多组磁盘（如RAID组）的聚合。 聚合具有自己的物理卷块数（pvbn）空间，并在该pvbn空间内维护诸如块分配结构之类的元数据。 每个vvol具有自己的虚拟卷块号（vvbn）空间，并在该vvbn空间内维护诸如块分配结构的元数据。 值得注意的是，vvol的块分配结构的大小适合于vvol，而不是基础聚合，从而允许管理由存储系统服务的数据（例如，快照操作）的操作有效地在vvols上工作。 文件系统布局扩展了常规写入任何文件布局系统实现的文件系统布局，但是保持了传统实现的性能特性。

公开(公告)号：US11340812B2

公开(公告)日：2022-05-24

申请号：US17064423

申请日：2020-10-06

Applicant: NETAPP, INC.

Inventor： Jayalakshmi Pattabiraman ,  Nikhil Mattankot ,  Deekshith Belchapada ,  Blake H. Lewis ,  Subramaniam Periyagaram ,  Satya Suresh Chouta Naga Veera ,  Rohit Singh ,  Rajesh Khandelwal ,  James Robert Morefield

IPC: G06F3/06 ,  G06F7/06 ,  G06F12/02

Abstract: A first plurality of block identifiers is sorted based, at least in part, on a measure of spatial locality. A second plurality of block identifiers is sorted based, at least in part, on the measure of spatial locality. At least the first plurality of block identifiers and the second plurality of block identifiers are incrementally merged into a third plurality of block identifiers based, at least in part, on the measure of spatial locality. A block of data corresponding to metadata associated with a plurality of block identifiers of the third plurality of block identifiers is updated.

公开(公告)号：US09448924B2

公开(公告)日：2016-09-20

申请号：US14150717

申请日：2014-01-08

Applicant: NetApp, Inc.

Inventor： Rajesh Sundaram ,  Stephen Daniel ,  Jeffrey S. Kimmel ,  Blake H. Lewis

IPC: G06F12/00 ,  G06F12/02 ,  G06F3/06 ,  G06F17/30

CPC classification number: G06F17/30138 ,  G06F3/0604 ,  G06F3/0608 ,  G06F3/0616 ,  G06F3/064 ,  G06F3/0643 ,  G06F3/0644 ,  G06F3/0647 ,  G06F3/0659 ,  G06F3/0679 ,  G06F3/0685 ,  G06F3/0688 ,  G06F12/0246 ,  G06F17/30185 ,  G06F17/30218 ,  G06F2003/0697 ,  G06F2212/2022 ,  G06F2212/7208

Abstract: In one embodiment, storage arrays of solid state drives (SSDs) coupled to a node are organized as redundant array of independent disks (RAID) groups. Each storage array includes one or more segments. Each segment has contiguous free space on the SSDs. Data and metadata is organized on the SSDs with a sequential log-structured layout, with the data organized as variable-length extents of one or more logical units (LUNs). Segment cleaning is performed to clean a selected segment by moving the extents of the selected segment that contain valid data to one or more different segments so as to free the selected segment. Additional extents are written as a sequence of contiguous range write operations to the entire free segment with temporal locality to reduce data relocation within the SSDs as a result of the write operations.

Abstract translation: 在一个实施例中，耦合到节点的固态驱动器（SSD）的存储阵列被组织为独立磁盘（RAID）组的冗余阵列。 每个存储阵列包括一个或多个段。 每个段在SSD上具有连续的可用空间。 数据和元数据在具有顺序日志结构布局的SSD上组织，数据组织为一个或多个逻辑单元（LUN）的可变长度范围。 执行段清洁以通过将包含有效数据的所选段的范围移动到一个或多个不同段来清理所选择的段，以便释放所选择的段。 作为连续范围写入操作的序列，附加范围被写入具有时间局部性的整个空闲段，以减少作为写入操作的结果的SSD内的数据重定位。

公开(公告)号：US20160139838A1

公开(公告)日：2016-05-19

申请号：US14546214

申请日：2014-11-18

Applicant: NetApp, Inc.

Inventor： Janice D'Sa ,  Ling Zheng ,  Blake H. Lewis

IPC: G06F3/06

CPC classification number: G06F3/0619 ,  G06F3/0607 ,  G06F3/0626 ,  G06F3/0644 ,  G06F3/0665 ,  G06F3/067 ,  G06F3/0689 ,  H04L67/1097

Abstract: A three-way merge technique efficiently updates metadata in accordance with a three-way merge operation managed by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. The metadata is embodied as mappings from logical block addresses (LBAs) of a logical unit (LUN) accessible by a host to durable extent keys, and is organized as a multi-level dense tree. The mappings are organized such that a higher level of the dense tree contains more recent mappings than a next lower level, i.e., the level immediately below. The three-way merge operation is an efficient (i.e., optimized) way of updating the volume metadata mappings of the dense tree by merging the mapping content of all three levels in a single iteration, as opposed to merging the content of the first level with the content of the second level in a first iteration of a two-way merge operation and then merging the results of the first iteration with the content of the third level in a second iteration of the operation.

Abstract translation: 三向合并技术根据在集群的一个或多个节点上执行的存储输入/输出（I / O）栈的卷层管理的三向合并操作来有效地更新元数据。 元数据体现为从主机可访问的逻辑单元（LUN）的逻辑块地址（LBA）到持久扩展密钥的映射，并且被组织为多级密集树。 这些映射被组织成使得较高级别的密集树包含比下一个较低级别更近的映射，即紧接在下面的级别。 三次合并操作是通过在单次迭代中合并所有三个级别的映射内容来更新密集树的卷元数据映射的有效（即优化的）方法，而不是将第一级的内容与 在双向合并操作的第一迭代中的第二级的内容，然后在操作的第二次迭代中将第一次迭代的结果与第三级的内容合并。

公开(公告)号：US20160110392A1

公开(公告)日：2016-04-21

申请号：US14978290

申请日：2015-12-22

Applicant: NetApp, Inc.

Inventor： Ling Zheng ,  Blake H. Lewis ,  Kayuri H. Patel

IPC: G06F17/30

CPC classification number: G06F17/30327 ,  G06F3/0611 ,  G06F3/0619 ,  G06F3/0638 ,  G06F3/0641 ,  G06F3/0655 ,  G06F3/0665 ,  G06F3/067 ,  G06F3/0689 ,  G06F17/30 ,  G06F17/30292

Abstract: The embodiments described herein are directed to an organization of metadata managed by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. The metadata managed by the volume layer, i.e., the volume metadata, is illustratively embodied as mappings from addresses, i.e., logical block addresses (LBAs), of a logical unit (LUN) accessible by a host to durable extent keys maintained by an extent store layer of the storage I/O stack. In an embodiment, the volume layer organizes the volume metadata as a mapping data structure, i.e., a dense tree metadata structure, which represents successive points in time to enable efficient access to the metadata.

Abstract translation: 这里描述的实施例涉及由在集群的一个或多个节点上执行的存储输入/输出（I / O）栈的卷层管理的元数据的组织。 由卷层管理的元数据（即卷元数据）被示意性地体现为从主机可访问的逻辑单元（LUN）的地址（即，逻辑块地址（LBA））到可由扩展区维护的持久范围密钥的映射 存储I / O堆栈的存储层。 在一个实施例中，卷层将卷元数据组织为映射数据结构，即密集树元数据结构，其表示连续的时间点以实现对元数据的有效访问。

公开(公告)号：US09152335B2

公开(公告)日：2015-10-06

申请号：US14160216

申请日：2014-01-21

Applicant: NetApp, Inc.

Inventor： Rajesh Sundaram ,  Jeffrey S. Kimmel ,  Blake H. Lewis

IPC: G06F12/12 ,  G06F3/06

CPC classification number: G06F3/0641 ,  G06F3/0604 ,  G06F3/0608 ,  G06F3/0643 ,  G06F3/067 ,  G06F3/0688 ,  G06F2003/0697

Abstract: In one embodiment, a layered file system of a storage input/output (I/O) stack executes on one or more nodes of a cluster. The layered file system includes a flash-optimized, log-structured layer configured to provide sequential storage of data and metadata (i.e., a log-structured layout) on solid state drives (SSDs) of storage arrays in the cluster to reduce write amplification, while leveraging a data de-duplication feature of the storage I/O stack. An extent store layer of the file system performs and maintains mappings of the extent keys to SSD storage locations, while a volume layer of the file system performs and maintains mappings of the LUN offset ranges to the extent keys. Separation of the mapping functions between the volume and extent store layers enables different volumes with different offset ranges to reference a same extent key (and thus a same extent).

Abstract translation: 在一个实施例中，存储输入/输出（I / O）堆栈的分层文件系统在集群的一个或多个节点上执行。 分层文件系统包括闪存优化的日志结构化层，其被配置为在集群中的存储阵列的固态驱动器（SSD）上提供数据和元数据（即，日志结构化布局）的顺序存储以减少写入放大， 同时利用存储I / O堆栈的重复数据删除功能。 文件系统的扩展存储层执行并维护扩展密钥到SSD存储位置的映射，而文件系统的卷层执行并维护LUN偏移范围到扩展密钥的映射。 卷和卷存储层之间的映射函数的分离使得具有不同偏移范围的不同卷可以引用相同的扩展密钥（并因此相同的扩展）。

公开(公告)号：US20150193338A1

公开(公告)日：2015-07-09

申请号：US14150717

申请日：2014-01-08

Applicant: NetApp, Inc.

Inventor： Rajesh Sundaram ,  Stephen Daniel ,  Jeffrey S. Kimmel ,  Blake H. Lewis

IPC: G06F12/02 ,  G06F3/06

CPC classification number: G06F17/30138 ,  G06F3/0604 ,  G06F3/0608 ,  G06F3/0616 ,  G06F3/064 ,  G06F3/0643 ,  G06F3/0644 ,  G06F3/0647 ,  G06F3/0659 ,  G06F3/0679 ,  G06F3/0685 ,  G06F3/0688 ,  G06F12/0246 ,  G06F17/30185 ,  G06F17/30218 ,  G06F2003/0697 ,  G06F2212/2022 ,  G06F2212/7208

Abstract: In one embodiment, storage arrays of solid state drives (SSDs) coupled to a node are organized as redundant array of independent disks (RAID) groups. Each storage array includes one or more segments. Each segment has contiguous free space on the SSDs. Data and metadata is organized on the SSDs with a sequential log-structured layout, with the data organized as variable-length extents of one or more logical units (LUNs). Segment cleaning is performed to clean a selected segment by moving the extents of the selected segment that contain valid data to one or more different segments so as to free the selected segment. Additional extents are written as a sequence of contiguous range write operations to the entire free segment with temporal locality to reduce data relocation within the SSDs as a result of the write operations.

Abstract translation: 在一个实施例中，耦合到节点的固态驱动器（SSD）的存储阵列被组织为独立磁盘（RAID）组的冗余阵列。 每个存储阵列包括一个或多个段。 每个段在SSD上具有连续的可用空间。 数据和元数据在具有顺序日志结构布局的SSD上组织，数据组织为一个或多个逻辑单元（LUN）的可变长度范围。 执行段清洁以通过将包含有效数据的所选段的范围移动到一个或多个不同段来清理所选择的段，以便释放所选择的段。 作为连续范围写入操作的序列，附加范围被写入具有时间局部性的整个空闲段，以减少作为写入操作的结果的SSD内的数据重定位。

公开(公告)号：US20150193170A1

公开(公告)日：2015-07-09

申请号：US14150689

申请日：2014-01-08

Applicant: NetApp, Inc.

Inventor： Rajesh Sundaram ,  Jeffrey S. Kimmel ,  Blake H. Lewis

IPC: G06F3/06

CPC classification number: G06F3/0641 ,  G06F3/0604 ,  G06F3/0608 ,  G06F3/0643 ,  G06F3/067 ,  G06F3/0688 ,  G06F2003/0697

Abstract: In one embodiment, a layered file system includes a volume layer and an extent store layer configured to provide sequential log-structured layout of data and metadata on solid state drives (SSDs) of one or more storage arrays. The data is organized as variable-length extents of one or more logical units (LUNs). The metadata includes volume metadata mappings from offset ranges of a LUN to extent keys and extent metadata mappings of the extent keys to storage locations of the extents on the SSDs. The extent store layer maintaining the extent metadata mappings determines whether an extent is stored on a storage array, and, in response to determination that the extent is stored on the storage array, returns an extent key for the stored extent to the volume layer to enable global inline de-duplication that obviates writing a duplicate copy of the extent on the storage array.

Abstract translation: 在一个实施例中，分层文件系统包括卷层和盘区存储层，其被配置为在一个或多个存储阵列的固态驱动器（SSD）上提供数据和元数据的顺序日志结构布局。 数据被组织为一个或多个逻辑单元（LUN）的可变长度范围。 元数据包括从LUN的偏移范围到扩展密钥的卷元数据映射以及扩展密钥到SSD上的扩展区的存储位置的扩展元数据映射。 维护盘区元数据映射的盘区存储层确定盘区是否存储在存储阵列上，并且响应于确定该盘区存储在存储阵列上，将存储的盘区的盘区密钥返回到卷层以启用 全局内联重复数据删除，从而避免在存储阵列上写入扩展区的重复副本。