公开(公告)号：EP3051700A1

公开(公告)日：2016-08-03

申请号：EP16153016.7

申请日：2016-01-27

申请人： HGST Netherlands B.V.

发明人： Bandic, Zvonimir ,  Guyot, Cyril ,  Li, Dongyang ,  Narasimha, Ashwin ,  Wang, Qingbo ,  Yang , Ken

IPC分类号： H03M7/30

CPC分类号： G06F17/30303 ,  G06F17/30371 ,  H03M7/3091 ,  H03M7/3093 ,  H03M7/6029

摘要： An approach for fingerprinting large data objects at the wire speed has been disclosed. The techniques include Fresh/Shift pipelining, split Fresh, optimization, online channel sampling, and pipelined selection. The architecture can also be replicated to work in parallel for higher system throughput. Fingerprinting may provide an efficient mechanism for identifying duplication in a data stream, and deduplication based on the identified fingerprints may provide reduced storage costs, reduced network bandwidth consumption, reduced processing time and other benefits. In some embodiments, fingerprinting may be used to ensure or verify data integrity and may facilitate detection of corruption or tampering. An efficient manner of generating fingerprints (either via hardware, software, or a combination) may reduce a computation load and/or time required to generate fingerprints.

摘要翻译： 已经公开了以线速指纹大数据对象的方法。 这些技术包括Fresh / Shift流水线，分割新鲜，优化，在线渠道采样和流水线选择。 该架构也可以被复制为并行工作，以实现更高的系统吞吐量。 指纹可以提供用于识别数据流中的重复的有效机制，并且基于所识别的指纹的重复数据消除可以提供降低的存储成本，减少的网络带宽消耗，缩短的处理时间和其他益处。 在一些实施例中，指纹可用于确保或验证数据完整性，并且可以便于检测到损坏或篡改。 生成指纹（通过硬件，软件或组合）的有效方式可以减少生成指纹所需的计算负荷和/或时间。

公开(公告)号：EP2614592A1

公开(公告)日：2013-07-17

申请号：EP10752348.2

申请日：2010-09-09

申请人： Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.

发明人： MARPE, Detlev ,  SCHWARZ, Heiko ,  WIEGAND, Thomas

IPC分类号： H03M7/42

CPC分类号： H03M7/40 ,  H03M7/4081 ,  H03M7/42 ,  H03M7/6029 ,  H03M7/607

摘要： In PIPE coding, where alphabet symbols are distributed among a plurality of specialized entropy en/decoders according to their probability distribution estimate, a categorizing stage is provided where source symbols to be encoded are sub-divided into a first substream which is subject to VLC coding, and a second substream which is subject to PIPE coding. By this measure, source symbols having an appropriate symbol probability distribution, i.e. a probability distribution suitable for being efficiently coded by means of VLC coding without the deficiencies outlined above in the introductory portion of the specification of the present application, may be categorized as VLC coded symbols whereas other symbols may be treated as PIPE coded symbols and subject to PIPE coding, the coding complexity of which is higher than VLC coding, but at a better compression efficiency.

摘要翻译： 在PIPE编码中，其中字母符号根据其概率分布估计在多个专用熵en /解码器之间分配，提供分类阶段，其中待编码的源符号被细分为经受VLC编码的第一子流 以及受PIPE编码的第二子流。 通过这种措施，具有适当的符号概率分布的源符号，即适用于通过VLC编码有效地编码的概率分布，而没有上述在本申请的说明书的介绍部分中概述的缺陷，可以被分类为VLC编码 符号，而其他符号可以被视为PIPE编码符号并经受PIPE编码，其编码复杂度高于VLC编码，但是具有更好的压缩效率。