公开(公告)号：US08462781B2

公开(公告)日：2013-06-11

申请号：US13080906

申请日：2011-04-06

申请人： David W. McGhee ,  Stefan Johansson ,  Shardendu Pandey ,  Keith E. Chaney ,  Charles A. Webb, III

发明人： David W. McGhee ,  Stefan Johansson ,  Shardendu Pandey ,  Keith E. Chaney ,  Charles A. Webb, III

IPC分类号： H04L12/28 ,  H04L12/56

CPC分类号： H04L45/7453 ,  H04L43/028 ,  H04L43/12

摘要： Systems and methods are disclosed for in-line removal of duplicate network packets in network packet streams operating at high speeds (e.g., 1-10 Gbps and above). A hash generator applies at least one hash algorithm to incoming packets to form one or more different hash values. The packet deduplication systems and methods then use the one or more hash values for each incoming packet to identify data stored for previously received backs and use the identified data to determine if incoming packets are duplicate packets. Duplicate packets are then removed from the output packet stream thereby reducing duplicate packets for downstream processing. A deduplication window can further be utilized to limit the amount of data stored for previous packets based upon one or more parameters, such as an amount of time that has passed and/or a number of packets for which data has been stored. These parameters can also be selected, configured and/or adjusted to achieve desired operational objectives.

摘要翻译： 公开的系统和方法用于在以高速（例如，1-10Gbps及更高）运行的网络分组流中在线去除重复的网络分组。 散列生成器将至少一个散列算法应用于传入数据包以形成一个或多个不同的哈希值。 然后，分组重复数据删除系统和方法使用每个输入分组的一个或多个散列值来识别为先前接收的背部存储的数据，并使用所识别的数据来确定传入分组是否是重复的分组。 然后从输出分组流中删除重复的分组，从而减少用于下游处理的重复分组。 可以进一步利用重复数据消除窗口来基于一个或多个参数（例如已经经过的时间量和/或数据已被存储的数据包的数量）来限制对先前分组所存储的数据量。 也可以选择，配置和/或调整这些参数以实现所需的操作目标。