Encrypting data objects in a data storage system

    公开(公告)号:US10146703B1

    公开(公告)日:2018-12-04

    申请号:US14984185

    申请日:2015-12-30

    申请人: EMC Corporation

    IPC分类号: G06F12/14 G06F3/06 H04L9/06

    摘要: Techniques for providing encryption of individual data objects in a data storage system include realizing data objects in the form of container files stored in a set of file systems, and encrypting individual ones of the data objects by encrypting the container files realizing the data objects using encryption keys associated with the individual data objects. By independently encrypting the container files that realize individual data objects, the disclosed system provides per-data object encryption. Each data object may be encrypted differently, e.g. using a different encryption key, even when multiple data objects are hosted over the same storage device or over a shared set of storage devices.

    Recovering performance of a file system post-migration
    2.
    发明授权
    Recovering performance of a file system post-migration 有权
    恢复文件系统迁移后的性能

    公开(公告)号:US09122689B1

    公开(公告)日:2015-09-01

    申请号:US13832695

    申请日:2013-03-15

    申请人: EMC Corporation

    IPC分类号: G06F17/30

    CPC分类号: G06F17/30079 G06F17/30067

    摘要: An improved technique involves providing a pair of inode numbers in a directory entry for a file in a directory of a target file system. Along these lines, the target file system writes a first inode number and a second inode number to the directory entry for the file. While the client has not yet remounted its file system, that client may continue to use the file handle provided for the original file system. The data storage system continues to refer to a mapping table that cross-references initial inode numbers to target ones, at some cost in performance. Nevertheless, the client has an option to remount the file system at a time that is convenient to the client, so that the client may obtain a new handle. The client may subsequently use the new file handle to access the file.

    摘要翻译: 改进的技术涉及在目标文件系统的目录中的文件的目录条目中提供一对索引节点号。 按照这些方式,目标文件系统将第一个inode号和第二个inode号写入文件的目录条目。 当客户端尚未重新安装其文件系统时,该客户端可能会继续使用为原始文件系统提供的文件句柄。 数据存储系统继续参考映射表,其将初始的inode编号交叉引用到目标索引节点号码,性能上有一些代价。 然而,客户端可以选择在客户端方便的时间重新安装文件系统,以便客户端可以获得一个新的句柄。 客户端可以随后使用新的文件句柄来访问该文件。

    Metadata flushing using additive increase and multiplicitive decrease to determine a number of IO processes used
    4.
    发明授权
    Metadata flushing using additive increase and multiplicitive decrease to determine a number of IO processes used 有权
    使用添加剂增加和乘数减少的元数据冲洗以确定使用的IO进程的数量

    公开(公告)号:US09280467B1

    公开(公告)日:2016-03-08

    申请号:US13834360

    申请日:2013-03-15

    申请人: EMC Corporation

    IPC分类号: G06F12/08 G06F3/06

    摘要: A method and a system to dynamically determine how much of the total IO bandwidth may be used for flushing dirty metadata from the cache to the main memory without increasing the host memory access latency time, includes increasing the number of IO processes by adding a number of IO processes at short intervals and measuring host latency. If the host latency is acceptable, then increasing the number of IO processes again by the same number, and repeating until the host latency period reaches a limit. When the limit has been reached, reducing the number of IO processes by a multiplicative factor, and repeating the additive process from the reduced number of IO processes. The number of IO processes used for flushing dirty metadata may resemble a series of saw teeth, rising gradually and declining rapidly in response to the number of host IO processes needed.

    摘要翻译: 一种方法和系统,用于动态地确定可以将多少总IO带宽用于将脏元数据从高速缓存刷新到主存储器,而不增加主机存储器访问延迟时间,包括通过添加若干个 IO处理时间短,测量主机延迟。 如果主机延迟可以接受,则再次增加IO进程的数量相同的数量,并重复,直到主机等待时间达到极限。 当达到极限时,通过乘法因子减少IO进程的数量,并从减少的IO进程数重复加法处理。 用于刷新脏元数据的IO进程的数量可能类似于一系列锯齿,其响应于所需的主机IO进程的数量逐渐上升并且迅速下降。

    Techniques for performing slice consistency checks
    5.
    发明授权
    Techniques for performing slice consistency checks 有权
    执行切片一致性检查的技术

    公开(公告)号:US09262424B1

    公开(公告)日:2016-02-16

    申请号:US13837751

    申请日:2013-03-15

    申请人: EMC Corporation

    IPC分类号: G06F7/00 G06F17/30

    摘要: A slice consistency check method is presented including, for each slice of a plurality of slices (a) obtaining a first vote for ownership of that slice according to a filesystem manager configured to manage a set of filesystems stored on a set of sparse metavolumes, (b) obtaining a second vote for the ownership of that slice according to a first driver, the first driver being independent of the filesystem manager, (c) obtaining a third vote for the ownership of that slice according to a second driver, the second driver being independent of the filesystem manager and the first driver, and (d) assigning ownership of that slice based on a majority vote of the first vote for the ownership of that slice, the second vote for the ownership of that slice, and the third vote for the ownership of that slice.

    摘要翻译: 提出了一种切片一致性检查方法,其包括对于多个切片的每个切片(a),根据被配置为管理存储在一组稀疏metavolumes上的一组文件系统的文件系统管理器获得该切片的所有权的第一投票(a) b)根据第一驱动程序获得对所述切片的所有权的第二投票,所述第一驱动程序独立于所述文件系统管理器,(c)根据第二驱动程序获得对所述切片的所有权的第三投票,所述第二驱动程序 独立于文件系统管理员和第一个驱动程序,以及(d)基于对该片段的所有权的第一次投票的多数投票,对该片段的所有权的第二投票以及第三投票来分配该片段的所有权 为该片的所有权。

    Unified data protection for block and file objects
    6.
    发明授权
    Unified data protection for block and file objects 有权
    块和文件对象的统一数据保护

    公开(公告)号:US09280555B1

    公开(公告)日:2016-03-08

    申请号:US13853508

    申请日:2013-03-29

    申请人: EMC Corporation

    IPC分类号: G06F17/30

    摘要: A technique for protecting host data using, for example, snaps, asynchronous replication, and/or synchronous replication, includes storing both block-based objects and file-based objects in a common form—as files. With both block-based objects and file-based objects represented as the same type of underlying objects, data protection of both block-based and file-based objects is accomplished using a single set of data protection technologies, which are configured to perform data protection operations on files.

    摘要翻译: 使用例如快照,异步复制和/或同步复制来保护主机数据的技术包括以通用形式存储基于块的对象和基于文件的对象作为文件。 使用基于块的对象和基于文件的对象表示为相同类型的底层对象,基于块和基于文件的对象的数据保护是使用一组数据保护技术完成的,这些数据保护技术被配置为执行数据保护 文件操作。

    Managing metadata synchronization for reducing host system latency in a storage system
    7.
    发明授权
    Managing metadata synchronization for reducing host system latency in a storage system 有权
    管理元数据同步,以减少存储系统中的主机系统延迟

    公开(公告)号:US09229864B1

    公开(公告)日:2016-01-05

    申请号:US13834385

    申请日:2013-03-15

    申请人: EMC Corporation

    IPC分类号: G06F12/08

    摘要: Flushing cache memory of dirty metadata in a plurality of file systems without either letting the caches reach their maximum capacity, or using so much of the total system IO process bandwidth that host system IO process requests are unreasonably delayed, may include determining the length of an interval between sync operations for each individual one of the plurality of file system, and how to divide a system wide maximum sync process IO operation bandwidth fairly between various ones of the plurality of file systems. A computer dynamically measures overall system operation rates, and calculates an available portion of a current calculated sync operation bandwidth for each file system. The computer also measures file system operation rates and determines how long a time period should be between sync operations in each file system.

    摘要翻译: 在多个文件系统中刷新脏元数据的高速缓冲存储器,而不用让缓存达到其最大容量,或者使用主机系统IO处理请求无法延迟的整个系统IO进程带宽的大部分可能包括确定 多个文件系统中的每一个文件系统的同步操作之间的间隔,以及如何在多个文件系统中的各个文件系统之间公平地划分系统范围的最大同步处理IO操作带宽。 计算机动态地测量整个系统操作速率,并计算每个文件系统当前计算的同步操作带宽的可用部分。 计算机还测量文件系统操作速率,并确定每个文件系统中的同步操作之间的时间长度多长时间。

    Storage pool-backed file system expansion

    公开(公告)号:US09940332B1

    公开(公告)日:2018-04-10

    申请号:US14317931

    申请日:2014-06-27

    申请人: EMC Corporation

    IPC分类号: G06F17/30

    CPC分类号: G06F17/30091

    摘要: Described are methods, systems, and apparatus, including computer program products for expanding a storage pool-backed file system. A file system expansion trigger is detected. First reserved space in the storage pool is reserved. A LUN size attribute associated with a LUN underlying the file system is increased. A file system size attribute associated with the file system is increased. First data to write to the file system is received. In response to receiving the first data, an expansion result is determined, wherein the expansion result indicates whether to expand the LUN and the file system. If the expansion result indicates to expand the LUN and the file system, a first portion of the first reserved space is provisioned and formatted. The first data is written to the file system.

    Unified data services for block and file objects
    9.
    发明授权
    Unified data services for block and file objects 有权
    块和文件对象的统一数据服务

    公开(公告)号:US09122697B1

    公开(公告)日:2015-09-01

    申请号:US13853284

    申请日:2013-03-29

    申请人: EMC Corporation

    IPC分类号: G06F17/30

    CPC分类号: G06F17/30091

    摘要: An improved technique for a data storage apparatus that combines both block-based and file-based functionality in a unified data path architecture. The improved technique brings together IO processing of block-based storage systems and file-based storage systems by expressing both block-based objects (e.g., LUNs) and file-based objects (e.g., host file systems) in the form of files. These files are parts of an underlying set of internal file systems stored on a set of storage units served by a storage pool. Because block and file-based objects are all expressed as files of this set of internal file systems, a common set of services can be applied across block-based and file-based objects. In particular, enhanced data services such as compression, automated storage tiering and deduplication are provided across both types of object using one set of common mechanisms.

    摘要翻译: 一种用于数据存储装置的改进技术,其在统一的数据路径架构中组合基于块的和基于文件的功能。 改进的技术通过以文件的形式表达基于块的对象(例如,LUN)和基于文件的对象(例如,主机文件系统)来汇集基于块的存储系统和基于文件的存储系统的IO处理。 这些文件是存储在由存储池服务的一组存储单元上的一组内部文件系统的一部分。 因为基于块和文件的对象都表示为这组内部文件系统的文件,所以可以跨基于块和基于文件的对象应用一组通用的服务。 特别地,使用一组通用机制在两种类型的对象之间提供增强的数据服务,例如压缩,自动存储分层和重复数据消除。