Peer to peer network write deduplication

    公开(公告)号:US09842114B2

    公开(公告)日:2017-12-12

    申请号:US14865869

    申请日:2015-09-25

    Applicant: NetApp, Inc.

    CPC classification number: G06F17/3015 G06F17/30194 G06F17/30867

    Abstract: A technique described herein performs peer to peer network write deduplication. A host system generates a fingerprint for data associated with a write request. The host system may then determine whether the generated fingerprint matches a local fingerprint stored in a local data structure or whether the generated fingerprint matches a global fingerprint associated with a global data structure, wherein the local fingerprint is associated with data previously written to the storage system by the host and wherein the global fingerprint is associated with data previously written to the storage system by a different host. If a match is found, the host system constructs a deduplication command utilizing a logical address corresponding to a storage location that stores the data. If a match is not found, a write command for the data of the write request is constructed and sent to the storage system.

    EXTENSIBLE AND ELASTIC DATA MANAGEMENT SERVICES ENGINE EXTERNAL TO A STORAGE DOMAIN
    12.
    发明申请
    EXTENSIBLE AND ELASTIC DATA MANAGEMENT SERVICES ENGINE EXTERNAL TO A STORAGE DOMAIN 审中-公开
    可扩展和弹性数据管理服务引擎到外部存储域

    公开(公告)号:US20170031955A1

    公开(公告)日:2017-02-02

    申请号:US14815470

    申请日:2015-07-31

    Applicant: NetApp, Inc.

    CPC classification number: H04L67/142 H04L67/1097 H04L67/327

    Abstract: A data management services architecture includes architectural components that run in both a storage and compute domains. The architectural components redirect storage requests from the storage domain to the compute domain, manage resources allocated from the compute domain, ensure compliance with a policy that governs resource consumption, deploy program code for data management services, dispatch service requests to deployed services, and monitor deployed services. The architectural components also include a service map to locate program code for data management services, and service instance information for monitoring deployed services and dispatching requests to deployed services. Since deployed services can be stateless or stateful, the services architecture also includes state data for the stateful services, with supporting resources that can expand or contract based on policy and/or service demand. The architectural components also include containers for the deployed services.

    Abstract translation: 数据管理服务架构包括在存储和计算域中运行的架构组件。 架构组件将存储请求从存储域重定向到计算域,管理从计算域分配的资源,确保符合管理资源消耗的策略,部署数据管理服务的程序代码,向部署的服务发送服务请求,并监视 部署服务。 架构组件还包括用于定位用于数据管理服务的程序代码的服务映射,以及用于监视所部署的服务并向部署的服务发送请求的服务实例信息。 由于部署的服务可以是无状态的或有状态的,所以服务体系结构还包括有状态服务的状态数据,可以根据策略和/或服务需求扩展或合同的支持资源。 架构组件还包括部署服务的容器。

    ELASTIC, EPHEMERAL IN-LINE DEDUPLICATION SERVICE

    公开(公告)号:US20230086414A1

    公开(公告)日:2023-03-23

    申请号:US18071790

    申请日:2022-11-30

    Applicant: NetApp Inc.

    Abstract: A deduplication service can be provided to a storage domain from a services framework that expands and contracts to both meet service demand and to conform to resource management of a compute domain. The deduplication service maintains a fingerprint database and reference count data in compute domain resources, but persists these into the storage domain for use in the case of a failure or interruption of the deduplication service in the compute domain. The deduplication service responds to service requests from the storage domain with indications of paths in a user namespace and whether or not a piece of data had a fingerprint match in the fingerprint database. The indication of a match guides the storage domain to either store the piece of data into the storage backend or to reference another piece of data. The deduplication service uses the fingerprints to define paths for corresponding pieces of data.

    Extensible and elastic data management services engine external to a storage domain

    公开(公告)号:US11232160B2

    公开(公告)日:2022-01-25

    申请号:US16375942

    申请日:2019-04-05

    Applicant: NetApp Inc.

    Abstract: A data management services architecture includes architectural components that run in both a storage and compute domains. The architectural components redirect storage requests from the storage domain to the compute domain, manage resources allocated from the compute domain, ensure compliance with a policy that governs resource consumption, deploy program code for data management services, dispatch service requests to deployed services, and monitor deployed services. The architectural components also include a service map to locate program code for data management services, and service instance information for monitoring deployed services and dispatching requests to deployed services. Since deployed services can be stateless or stateful, the services architecture also includes state data for the stateful services, with supporting resources that can expand or contract based on policy and/or service demand. The architectural components also include containers for the deployed services.

    REDUCING WRITE TAIL LATENCY IN STORAGE SYSTEMS

    公开(公告)号:US20170262191A1

    公开(公告)日:2017-09-14

    申请号:US15064141

    申请日:2016-03-08

    Applicant: NetApp, Inc.

    Abstract: In order to reduce write tail latency, a storage system generates redundant write requests when performing a storage operation for an object. The storage operation is determined to be effectively complete when a minimum number of write requests have completed. For example, the storage system may generate twelve write requests and also generate four redundant write requests for a total of sixteen write requests. The storage system considers the object successfully stored once twelve of the sixteen writes complete successfully. To generate the redundant writes, the storage system may use replication or erasure coding. For replication, the storage system may issue a redundant write request for each of n chunks being written. For erasure coding, the storage system may use rateless codes which can generate unlimited number of parity chunks or use an n+k+k' erasure code which generates an additional k encoded chunks, in place of an n+k erasure code.

    PEER TO PEER NETWORK WRITE DEDUPLICATION

    公开(公告)号:US20170091183A1

    公开(公告)日:2017-03-30

    申请号:US14865869

    申请日:2015-09-25

    Applicant: NetApp, Inc.

    CPC classification number: G06F17/3015 G06F17/30194 G06F17/30867

    Abstract: A technique described herein performs peer to peer network write deduplication. A host system generates a fingerprint for data associated with a write request. The host system may then determine whether the generated fingerprint matches a local fingerprint stored in a local data structure or whether the generated fingerprint matches a global fingerprint associated with a global data structure, wherein the local fingerprint is associated with data previously written to the storage system by the host and wherein the global fingerprint is associated with data previously written to the storage system by a different host. If a match is found, the host system constructs a deduplication command utilizing a logical address corresponding to a storage location that stores the data. If a match is not found, a write command for the data of the write request is constructed and sent to the storage system.

Patent Agency Ranking