公开(公告)号：US20100157840A1

公开(公告)日：2010-06-24

申请号：US12317420

申请日：2008-12-22

Applicant: Subhabrata Sen ,  Lee Breslau ,  Nicholas Duffield ,  Yu Gu

Inventor： Subhabrata Sen ,  Lee Breslau ,  Nicholas Duffield ,  Yu Gu

IPC: G06F11/30

CPC classification number: H04L43/0835 ,  H04L41/0213 ,  H04L41/142 ,  H04L41/5009 ,  H04L43/026 ,  H04L43/103 ,  Y02D50/30

Abstract: A packet loss estimation technique is disclosed that utilizes the sampled flow level statistics that are routinely collected in operational networks, thereby obviating the need for any new router features or measurement infrastructure. The technique is specifically designed to handle the challenges of sampled flow-level aggregation such as information loss resulting from packet sampling, and generally comprises: receiving a first record of sampled packets for a flow from a first network element; receiving a second record of sampled packets for the flow from a second network element communicating with the first network element; correlating sampled packets from the flow at the first network element and the second network element to a measurement interval; and estimating the packet loss using a count of the sampled packets correlated to the measurement interval.

Abstract translation: 公开了一种利用在操作网络中常规收集的采样流量统计信息的分组丢失估计技术，从而避免了对任何新的路由器特征或测量基础设施的需要。 该技术专门设计用于处理采样流级聚合的挑战，例如由分组采样导致的信息丢失，并且通常包括：从第一网络元件接收流的第一采样分组记录; 从与第一网络元件通信的第二网络元件接收用于流的采样分组的第二记录; 将来自第一网元和第二网元的流的采样分组相关联到测量间隔; 以及使用与测量间隔相关联的采样分组的计数来估计分组丢失。

公开(公告)号：US20100153316A1

公开(公告)日：2010-06-17

申请号：US12568044

申请日：2009-09-28

Applicant: Nicholas Duffield ,  Patrick Haffner ,  Balachander Krishnamurthy ,  Haakon Andreas Ringberg

Inventor： Nicholas Duffield ,  Patrick Haffner ,  Balachander Krishnamurthy ,  Haakon Andreas Ringberg

IPC: G06F21/00 ,  G06F15/18

CPC classification number: H04L63/20 ,  G06F21/552 ,  G06N5/025 ,  H04L41/00 ,  H04L41/0604 ,  H04L41/16 ,  H04L43/026 ,  H04L43/028 ,  H04L43/16 ,  H04L45/28 ,  H04L45/38 ,  H04L47/10 ,  H04L47/24 ,  H04L63/14 ,  H04L63/1416 ,  H04L63/1425 ,  H04L63/30

Abstract: A system to detect anomalies in internet protocol (IP) flows uses a set of machine-learning (ML) rules that can be applied in real time at the IP flow level. A communication network has a large number of routers that can be equipped with flow monitoring capability. A flow collector collects flow data from the routers throughout the communication network and provides them to a flow classifier. At the same time, a limited number of locations in the network monitor data packets and generate alerts based on packet data properties. The packet alerts and the flow data are provided to a machine learning system that detects correlations between the packet-based alerts and the flow data to thereby generate a series of flow-level alerts. These rules are provided to the flow time classifier. Over time, the new packet alerts and flow data are used to provide updated rules generated by the machine learning system.

Abstract translation: 检测互联网协议（IP）流中的异常的系统使用一组机器学习（ML）规则，可以在IP流级别实时应用。 通信网络具有大量可配备流量监控功能的路由器。 集流器在通信网络中收集来自路由器的流数据，并将其提供给流分类器。 同时，网络中有限数量的位置监视数据包，并根据数据包数据属性生成警报。 分组警报和流数据被提供给机器学习系统，其检测基于分组的警报和流数据之间的相关性，从而生成一系列流级别警报。 这些规则提供给流时间分类器。 随着时间的推移，新的数据包警报和流数据用于提供机器学习系统生成的更新规则。

公开(公告)号：US20100138529A1

公开(公告)日：2010-06-03

申请号：US12325340

申请日：2008-12-01

Applicant: Nicholas Duffield ,  Carsten Lund ,  Mikkel Thorup ,  Edith Cohen ,  Haim Kaplan

Inventor： Nicholas Duffield ,  Carsten Lund ,  Mikkel Thorup ,  Edith Cohen ,  Haim Kaplan

IPC: G06F17/18 ,  G06F15/16

CPC classification number: G06F17/18 ,  H04L41/142 ,  H04L43/024 ,  H04L43/16

Abstract: The present invention relates to a method of obtaining a generic sample of an input stream. The method is designated as VAROPTk. The method comprises receiving an input stream of items arriving one at a time, and maintaining a sample S of items i. The sample S has a capacity for at most k items i. The sample S is filled with k items i. An nth item i is received. It is determined whether the nth item i should be included in sample S. If the nth item i is included in sample S, then a previously included item i is dropped from sample S. The determination is made based on weights of items without distinguishing between previously included items i and the nth item i. The determination is implemented thereby updating weights of items i in sample S. The method is repeated until no more items are received.

Abstract translation: 本发明涉及一种获得输入流的通用样本的方法。 该方法被指定为VAROPTk。 该方法包括一次接收一个物品的输入流，并且保持项目i的样本S. 样本S具有最多k个项目i的容量。 样本S填充有k个项目i。 收到第n项。 确定第n个项目i是否应该包含在样本S中。如果第n个项目i包括在样本S中，则先前包括的项目i从样本S中丢弃。根据项目的权重进行确定，而不区分 以前包括项目i和第n项目i。 由此实现确定，从而更新样本S中的项目i的权重。重复该方法，直到不再收到项目。

公开(公告)号：US20090316590A1

公开(公告)日：2009-12-24

申请号：US12119939

申请日：2008-05-13

Applicant: Carsten Lund ,  Edith Cohen ,  Nicholas Duffield ,  Alexandre Gerber ,  Adam Hersh ,  Oliver Spatscheck ,  Mikkel Thorup ,  Frederick True

Inventor： Carsten Lund ,  Edith Cohen ,  Nicholas Duffield ,  Alexandre Gerber ,  Adam Hersh ,  Oliver Spatscheck ,  Mikkel Thorup ,  Frederick True

IPC: H04L12/26

CPC classification number: H04L43/026 ,  H04L43/022

Abstract: The preferred embodiments of the present invention can include sampling packets transmitted over a network based on the content of the packets. If a packet is sampled, the sampling unit can add one or more fields to the sampled packet that can include a field for a number of bytes contained in the packet, a packet count, a flow count, a sampling type, and the like. The sampled packets can be analyzed to discern desired information from the packets. The additional fields that are added to the sampled packets can be used during the analysis.

Abstract translation: 本发明的优选实施例可以包括基于分组的内容对通过网络传输的分组进行抽样。 如果分组被采样，则采样单元可以将一个或多个字段添加到采样分组，该分组可以包括分组中包含的多个字节的字段，分组计数，流量计数，采样类型等。 可以分析采样的分组以从分组中辨别所需的信息。 在分析过程中可以使用添加到采样数据包的附加字段。

公开(公告)号：US20090190487A1

公开(公告)日：2009-07-30

申请号：US12022733

申请日：2008-01-30

Applicant: Lee Breslau ,  Nicholas Duffield ,  Yu Gu ,  Subhabrata Sen

Inventor： Lee Breslau ,  Nicholas Duffield ,  Yu Gu ,  Subhabrata Sen

IPC: H04L12/26

CPC classification number: H04L12/4633 ,  H04L43/0864

Abstract: A method and apparatus for providing performance measurements on network tunnels in packet networks are disclosed. For example, the method establishes two tunnels between a first measurement host and a first router, and establishes a tunnel between the first router and a second measurement host. The method also establishes a multicast group having a plurality of members, and sends one or more packets addressed to the multicast group from the first measurement host. The method measures the frequencies of directly and/or indirectly received responses from the plurality of members of the multicast group, and provides a plurality of estimated values for a plurality of packet transmission rates from measurement of the frequencies for one or more of said tunnels.

Abstract translation: 公开了一种用于在分组网络中的网络隧道上提供性能测量的方法和装置。 例如，该方法在第一测量主机和第一路由器之间建立两个隧道，并在第一路由器和第二测量主机之间建立隧道。 该方法还建立具有多个成员的多播组，并且从第一测量主机发送寻址到多播组的一个或多个分组。 该方法测量来自多播组的多个成员的直接和/或间接接收的响应的频率，并从测量一个或多个所述隧道的频率提供多个分组传输速率的多个估计值。

公开(公告)号：US09349102B2

公开(公告)日：2016-05-24

申请号：US13620668

申请日：2012-09-14

Applicant: Subhabrata Sen ,  Nicholas Duffield ,  Patrick Haffner ,  Jeffrey Erman ,  Yu Jin

Inventor： Subhabrata Sen ,  Nicholas Duffield ,  Patrick Haffner ,  Jeffrey Erman ,  Yu Jin

IPC: G06N99/00

CPC classification number: G06N99/005

Abstract: A traffic classifier has a plurality of binary classifiers, each associated with one of a plurality of calibrators. Each calibrator trained to translate an output score of the associated binary classifier into an estimated class probability value using a fitted logistic curve, each estimated class probability value indicating a probability that the packet flow on which the output score is based belongs to the traffic class associated with the binary classifier associated with the calibrator. The classifier training system configured to generate a training data based on network information gained using flow and packet sampling methods. In some embodiments, the classifier training system configured to generate reduced training data sets, one for each traffic class, reducing the training data related to traffic not associated with the traffic class.

Abstract translation: 流量分类器具有多个二进制分类器，每个二进制分类器与多个校准器之一相关联。 每个校准器被训练成使用拟合的逻辑曲线将相关联的二进制分类器的输出得分转换成估计的类概率值，每个估计的类概率值指示输出得分所基于的分组流的概率属于相关联的流量类别 与校准器相关联的二进制分类器。 分类器训练系统被配置为基于使用流和分组采样方法获得的网络信息生成训练数据。 在一些实施例中，分类器训练系统被配置为生成减少的训练数据集，每个业务类别一个，减少与业务类别不相关的业务相关的训练数据。

公开(公告)号：US08935188B2

公开(公告)日：2015-01-13

申请号：US12858303

申请日：2010-08-17

Applicant: Nicholas Duffield ,  Patrick Haffner ,  Yu Jin ,  Subhabrata Sen ,  Zhi-Li Zhang

Inventor： Nicholas Duffield ,  Patrick Haffner ,  Yu Jin ,  Subhabrata Sen ,  Zhi-Li Zhang

IPC: G06F15/18 ,  G06F15/173 ,  H04L12/26 ,  H04L29/08

CPC classification number: H04L43/045 ,  H04L43/026 ,  H04L67/22

Abstract: In one embodiment, the present disclosure is a method and apparatus for classifying applications using the collective properties of network traffic. In one embodiment, a method for classifying traffic in a communication network includes receiving a traffic activity graph, the traffic activity graph comprising a plurality of nodes interconnected by a plurality of edges, where each of the nodes represents an endpoint associated with the communication network and each of the edges represents traffic between a corresponding pair of the nodes, generating an initial set of inferences as to an application class associated with each of the edges, based on at least one measured statistic related to at least one traffic flow in the communication network, and refining the initial set of inferences based on a spatial distribution of the traffic flows, to produce a final traffic activity graph.

Abstract translation: 在一个实施例中，本公开是用于使用网络业务的集合属性对应用进行分类的方法和装置。 在一个实施例中，用于对通信网络中的业务进行分类的方法包括接收业务活动图，所述业务活动图包括由多个边缘互连的多个节点，其中每个节点表示与所述通信网络相关联的端点， 每个边缘表示对应的一对节点之间的流量，基于与通信网络中的至少一个业务流相关的至少一个测量的统计量，生成关于与每个边缘相关联的应用类别的初始推断集合 ，并且基于业务流的空间分布来优化初始推理集合，以产生最终业务活动图。

公开(公告)号：US08195710B2

公开(公告)日：2012-06-05

申请号：US12653831

申请日：2009-12-18

Applicant: Edith Cohen ,  Nicholas Duffield ,  Haim Kaplan ,  Carsten Lund ,  Mikkel Thorup

Inventor： Edith Cohen ,  Nicholas Duffield ,  Haim Kaplan ,  Carsten Lund ,  Mikkel Thorup

IPC: G06F17/00

CPC classification number: H04L43/028 ,  H04L43/04

Abstract: A method for producing a summary A of data points in an unaggregated data stream wherein the data points are in the form of weighted keys (a, w) where a is a key and w is a weight, and the summary is a sample of k keys a with adjusted weights wa. A first reservoir L includes keys having adjusted weights which are additions of weights of individual data points of included keys and a second reservoir T includes keys having adjusted weights which are each equal to a threshold value τ whose value is adjusted based upon tests of new data points arriving in the data stream. The summary combines the keys and adjusted weights of the first reservoir L with the keys and adjusted weights of the second reservoir T to form the sample representing the data stream upon which further analysis may be performed. The method proceeds by first merging new data points in the stream into the reservoir L until the reservoir contains k different keys and thereafter applying a series of tests to new arriving data points to determine what keys and weights are to be added to or removed the reservoirs L and T to provide a summary with a variance that approaches the minimum possible for aggregated data sets. The method is composable, can be applied to high speed data streams such as those found on the Internet, and can be implemented efficiently.

Abstract translation: 一种用于产生未聚集数据流中的数据点的摘要A的方法，其中数据点是加权密钥（a，w）的形式，其中a是密钥，w是权重，并且摘要是k的样本 键a与调整权重wa。 第一储存器L包括具有调整权重的密钥，这些密钥是附加密钥的各个数据点的加权的加法，而第二储存器T包括具有调整的权重的密钥，其各自等于基于新数据的测试来调整其值的阈值τ 到达数据流的点。 总结将第一储层L的密钥和调整的权重与密钥和第二储存器T的调整权重组合，以形成表示可以进行进一步分析的数据流的样本。 该方法通过首先将流中的新数据点合并到储存器L中，直到储存器包含k个不同的密钥，然后对新的到达数据点应用一系列测试，以确定要添加到或移除存储器的哪些密钥和权重 L和T提供一个总结，其方差接近汇总数据集的最小可能性。 该方法是可组合的，可以应用于诸如在因特网上发现的高速数据流，并且可以有效地实现。

公开(公告)号：US08001601B2

公开(公告)日：2011-08-16

申请号：US11452623

申请日：2006-06-14

Applicant: Nicholas Duffield ,  Jacobus Van Der Merwe ,  Vyas Sekar ,  Oliver Spatscheck

Inventor： Nicholas Duffield ,  Jacobus Van Der Merwe ,  Vyas Sekar ,  Oliver Spatscheck

IPC: G06F11/00 ,  G06F12/14 ,  G06F12/16 ,  G08B23/00

CPC classification number: H04L63/1425 ,  H04L63/1458

Abstract: A multi-staged framework for detecting and diagnosing Denial of Service attacks is disclosed in which a low-cost anomaly detection mechanism is first used to collect coarse data, such as may be obtained from Simple Network Management Protocol (SNMP) data flows. Such data is analyzed to detect volume anomalies that could possibly be indicative of a DDoS attack. If such an anomaly is suspected, incident reports are then generated and used to trigger the collection and analysis of fine grained data, such as that available in Netflow data flows. Both types of collection and analysis are illustratively conducted at edge routers within the service provider network that interface customers and customer networks to the service provider. Once records of the more detailed information have been retrieved, they are examined to determine whether the anomaly represents a distributed denial of service attack, at which point an alarm is generated.

Abstract translation: 公开了一种用于检测和诊断拒绝服务攻击的多阶段框架，其中首先使用低成本异常检测机制来收集粗略数据，例如可以从简单网络管理协议（SNMP）数据流中获得。 分析这些数据以检测可能表示DDoS攻击的体积异常。 如果怀疑出现这种异常，则会生成事件报告，并用于触发对Netflow数据流中可用的细粒度数据的收集和分析。 这两种类型的收集和分析在服务提供商网络中的边缘路由器上进行说明性地进行，其将客户和客户网络接入服务提供商。 一旦检索到更详细信息的记录，就检查它们以确定异常是否表示分布式拒绝服务攻击，此时产生警报。

公开(公告)号：US20110040706A1

公开(公告)日：2011-02-17

申请号：US12539430

申请日：2009-08-11

Applicant: Subhabrata Sen ,  Nicholas Duffield ,  Patrick Haffner ,  Jeffrey Erman ,  Yu Jin

Inventor： Subhabrata Sen ,  Nicholas Duffield ,  Patrick Haffner ,  Jeffrey Erman ,  Yu Jin

IPC: G06F15/18 ,  G06N5/02

CPC classification number: G06N99/005

Abstract: A traffic classifier has a plurality of binary classifiers, each associated with one of a plurality of calibrators. Each calibrator trained to translate an output score of the associated binary classifier into an estimated class probability value using a fitted logistic curve, each estimated class probability value indicating a probability that the packet flow on which the output score is based belongs to the traffic class associated with the binary classifier associated with the calibrator. The classifier training system configured to generate a training data based on network information gained using flow and packet sampling methods. In some embodiments, the classifier training system configured to generate reduced training data sets, one for each traffic class, reducing the training data related to traffic not associated with the traffic class.

Abstract translation: 流量分类器具有多个二进制分类器，每个二进制分类器与多个校准器之一相关联。 每个校准器被训练成使用拟合的逻辑曲线将相关联的二进制分类器的输出得分转换成估计的类概率值，每个估计的类概率值指示输出得分所基于的分组流的概率属于相关联的流量类别 与校准器相关联的二进制分类器。 分类器训练系统被配置为基于使用流和分组采样方法获得的网络信息生成训练数据。 在一些实施例中，分类器训练系统被配置为生成减少的训练数据集，每个业务类别一个，减少与业务类别不相关的业务相关的训练数据。