公开(公告)号：US11256424B2

公开(公告)日：2022-02-22

申请号：US16532817

申请日：2019-08-06

Applicant: NetApp, Inc.

Inventor： Suganthi Dewakar ,  Xing Lin ,  Junji Zhi ,  Deepak Raghu Kenchammana-Hosekote

IPC: G11C29/00 ,  G06F3/06 ,  G06F11/10 ,  H03M13/15

Abstract: In order to reduce write tail latency, a storage system generates redundant write requests when performing a storage operation for an object. The storage operation is determined to be effectively complete when a minimum number of write requests have completed. For example, the storage system may generate twelve write requests and also generate four redundant write requests for a total of sixteen write requests. The storage system considers the object successfully stored once twelve of the sixteen writes complete successfully. To generate the redundant writes, the storage system may use replication or erasure coding. For replication, the storage system may issue a redundant write request for each of n chunks being written. For erasure coding, the storage system may use rateless codes which can generate unlimited number of parity chunks or use an n+k+k′ erasure code which generates an additional k′ encoded chunks, in place of an n+k erasure code.

公开(公告)号：US20200081868A1

公开(公告)日：2020-03-12

申请号：US16247014

申请日：2019-01-14

Applicant: NetApp, Inc.

Inventor： Xing Lin ,  Fan Ni

IPC: G06F16/174 ,  G06F16/14

Abstract: Methods, non-transitory machine readable media, and computing devices that compare a hash value to a predefined value for sliding windows in parallel for segments partitioned from an input data stream. A bit array is parsed according to minimum and maximum chunk sizes to identify chunk boundaries for the input data stream. The bit array is populated based on a result of the comparison and portions of the bit array are parsed in parallel. Unique chunks of the input data stream defined by the chunk boundaries are stored in a storage device. Accordingly, this technology utilizes parallel processing in two stages. In a first stage, rolling window based hashing is performed concurrently to identify potential chunk boundaries. In a second stage, actual chunk boundaries are selected based on minimum and maximum chunk size constraints. This technology advantageously facilitates significant deduplication ratio improvement as well as improved parallel chunking performance.

公开(公告)号：US20170262191A1

公开(公告)日：2017-09-14

申请号：US15064141

申请日：2016-03-08

Applicant: NetApp, Inc.

Inventor： Suganthi Dewakar ,  Xing Lin ,  Junji Zhi ,  Deepak Raghu Kenchammana-Hosekote

IPC: G06F3/06 ,  H03M13/15 ,  G06F11/10

CPC classification number: G06F3/0611 ,  G06F3/0659 ,  G06F3/0689 ,  G06F11/1076 ,  H03M13/154

Abstract: In order to reduce write tail latency, a storage system generates redundant write requests when performing a storage operation for an object. The storage operation is determined to be effectively complete when a minimum number of write requests have completed. For example, the storage system may generate twelve write requests and also generate four redundant write requests for a total of sixteen write requests. The storage system considers the object successfully stored once twelve of the sixteen writes complete successfully. To generate the redundant writes, the storage system may use replication or erasure coding. For replication, the storage system may issue a redundant write request for each of n chunks being written. For erasure coding, the storage system may use rateless codes which can generate unlimited number of parity chunks or use an n+k+k' erasure code which generates an additional k encoded chunks, in place of an n+k erasure code.

公开(公告)号：US20210409467A1

公开(公告)日：2021-12-30

申请号：US17471437

申请日：2021-09-10

Applicant: NetApp, Inc.

Inventor： Xing Lin

IPC: H04L29/06 ,  H04L12/24 ,  H04N1/64 ,  H04N19/103 ,  H04N1/32 ,  H04N19/42 ,  H04L29/08 ,  H04N19/70 ,  H04N19/00

Abstract: Methods, non-transitory machine readable media, and computing devices that provide improved dictionary-based compression are disclosed. With this technology, a first portion of an input data stream is compressed using a first dictionary. A second dictionary is trained when the first dictionary is determined to be stale. The dictionary can be determined to be stale based on a size of the input data stream compressed using the first dictionary or a compression ratio decreasing by a threshold, for example. The first dictionary can be stored with metadata associated with the compressed first portion of the input data stream. Accordingly, this technology improves compression ratios, eliminates the need for reference counting, and facilitates improved reclamation of orphan dictionaries, among other advantages.

公开(公告)号：US11444997B2

公开(公告)日：2022-09-13

申请号：US17471437

申请日：2021-09-10

Applicant: NetApp, Inc.

Inventor： Xing Lin

IPC: H04L69/04 ,  H04L65/61 ,  H04L41/147 ,  H04L41/16 ,  H04N1/64 ,  H04N19/103 ,  H04N1/32 ,  H04N19/42 ,  H04N19/70 ,  H04N19/00 ,  H04L65/65 ,  H04L67/565 ,  H04L67/5651

Abstract: Methods, non-transitory machine readable media, and computing devices that provide improved dictionary-based compression are disclosed. With this technology, a first portion of an input data stream is compressed using a first dictionary. A second dictionary is trained when the first dictionary is determined to be stale. The dictionary can be determined to be stale based on a size of the input data stream compressed using the first dictionary or a compression ratio decreasing by a threshold, for example. The first dictionary can be stored with metadata associated with the compressed first portion of the input data stream. Accordingly, this technology improves compression ratios, eliminates the need for reference counting, and facilitates improved reclamation of orphan dictionaries, among other advantages.

公开(公告)号：US20160179581A1

公开(公告)日：2016-06-23

申请号：US14576719

申请日：2014-12-19

Applicant: NetApp, Inc.

Inventor： Gokul Soundararajan ,  Jingxin Feng ,  Xing Lin

IPC: G06F9/50 ,  G06F17/30

CPC classification number: G06F9/5033

Abstract: Systems, devices, and methods are described for performing content-aware task assignment. A resource manager in a distributed computing system can identify tasks associated with a file. Each task can involve processing multiple data blocks of the file (e.g., in parallel with other processing by other tasks). The resource manager can provide block identifiers for the blocks to each of multiple computing nodes. Each computing node can store a respective subset of the blocks in a respective cache storage medium. Each subset of blocks stored at a node can be identified from the block identifiers. The resource manager can assign the task to a selected one of the computing nodes. The task can be assigned based on the selected computing node having larger subset of the blocks than one or more other computing nodes in the distributed computing system. In some embodiments, computing nodes can de-duplicate cached data using block identifiers.

Abstract translation: 描述了用于执行内容感知任务分配的系统，设备和方法。 分布式计算系统中的资源管理器可以识别与文件相关的任务。 每个任务可以涉及处理文件的多个数据块（例如，与其他任务的其他处理并行）。 资源管理器可以为多个计算节点中的每一个提供块的块标识符。 每个计算节点可以将各个块的相应子集存储在相应的高速缓存存储介质中。 可以从块标识符识别存储在节点处的块的每个子集。 资源管理器可以将任务分配给所选择的一个计算节点。 可以基于所选择的具有比分布式计算系统中的一个或多个其他计算节点更大子块子集的计算节点来分配该任务。 在一些实施例中，计算节点可以使用块标识符来重新缓存数据。

公开(公告)号：US11122095B2

公开(公告)日：2021-09-14

申请号：US16821328

申请日：2020-03-17

Applicant: NetApp, Inc.

Inventor： Xing Lin

IPC: H04L29/06 ,  H04L12/24 ,  H04N1/64 ,  H04N19/103 ,  H04N1/32 ,  H04N19/42 ,  H04L29/08 ,  H04N19/70 ,  H04N19/00

Abstract: Methods, non-transitory machine readable media, and computing devices that provide improved dictionary-based compression are disclosed. With this technology, a first portion of an input data stream is compressed using a first dictionary. A second dictionary is trained when the first dictionary is determined to be stale. The dictionary can be determined to be stale based on a size of the input data stream compressed using the first dictionary or a compression ratio decreasing by a threshold, for example. The first dictionary can be stored with metadata associated with the compressed first portion of the input data stream. Accordingly, this technology improves compression ratios, eliminates the need for reference counting, and facilitates improved reclamation of orphan dictionaries, among other advantages.

公开(公告)号：US20210092174A1

公开(公告)日：2021-03-25

申请号：US16821328

申请日：2020-03-17

Applicant: NetApp, Inc.

Inventor： Xing Lin

IPC: H04L29/06 ,  H04L12/24

Abstract: Methods, non-transitory machine readable media, and computing devices that provide improved dictionary-based compression are disclosed. With this technology, a first portion of an input data stream is compressed using a first dictionary. A second dictionary is trained when the first dictionary is determined to be stale. The dictionary can be determined to be stale based on a size of the input data stream compressed using the first dictionary or a compression ratio decreasing by a threshold, for example. The first dictionary can be stored with metadata associated with the compressed first portion of the input data stream. Accordingly, this technology improves compression ratios, eliminates the need for reference counting, and facilitates improved reclamation of orphan dictionaries, among other advantages.