摘要:
In one embodiment, a method of monitoring a network. The method includes: receiving, from each host of a set of two or more hosts of the network, a corresponding vector of M components constructed based on data packets received at the host during a time period, M being an integer greater than 1; and, based on the constructed vectors, using an expectation-maximization algorithm to estimate a cardinality distribution for the hosts in the set, wherein constructing a vector includes updating a component of the vector of the corresponding host in response to the corresponding host receiving a data packet, the updating including selecting the component for updating by hashing one or more fields of the data packet received by the corresponding host.
摘要:
A capability for incremental tracking of multiples quantiles is provided. A method for performing an incremental quantile update using a data value of a received data record includes determining an initial distribution function, updating the initial distribution function to form a new distribution function based on the received data value, generating an approximation of the new distribution function, and determining new quantile estimates from the approximation of the new distribution function. The initial distribution function includes a plurality of initial quantile estimates and a respective plurality of initial probabilities. The new distribution function includes a plurality of quantile points identifying the respective initial quantile estimates and a respective plurality of new probabilities associated with the initial quantile estimates. The approximation of the new distribution function is generated by connecting pairs of adjacent quantile points using linear approximations of regions between the pairs of adjacent quantile points.
摘要:
A network-equipment-implemented method and apparatus for tracking durations of flows received at a network node in consecutive intervals utilizes two counting bloom filters in ping-pong operation to reduce memory and processing. Identifiers for flows that exceed a predetermined duration or number of intervals are stored in a long-duration flow-identifier table. Hash functions used within the counting bloom filters and optionally used in the long-duration flow-identifier table are chosen to minimize the probability of false positives in the detection of long-duration flows. In some embodiments, flows are sampled to conserve memory and processing resources at the risk of missing detection of some long-duration flows.
摘要:
A method and apparatus are provided for incrementally tracking quantiles in the presence of multiple record types. A method for performing incremental quantile tracking includes receiving a first data record of a first record type having a first data value, determining whether a second data record of a second record type is received, determining an initial distribution function, updating the initial distribution function to form a new distribution function based on the first data value and whether a second data record is received, generating an approximation of the new distribution function, determining at least one new quantile estimate associated with at least one new probability of the new distribution function using the approximation of the new distribution function, and storing the at least one new quantile estimate and the at least one new probability associated with the at least one new quantile estimate.
摘要:
A method and apparatus are provided for incrementally tracking quantiles in the presence of multiple record types. A method for performing incremental quantile tracking includes receiving a first data record of a first record type and a second data record of a second record type, and updating a quantile probability for a quantile value, based on the first record type of the first data record and the second record type of the second data record, to obtain a new quantile probability for the quantile value.
摘要:
A method and apparatus for incremental tracking of multiples quantiles is provided. A method for performing an incremental quantile update using a data value of a received data record includes determining an initial distribution function, updating the initial distribution function to form a new distribution function based on the received data value, generating an approximation of the new distribution function, and determining new quantile estimates from the approximation of the new distribution function. The initial distribution function includes a plurality of initial quantile estimates and a respective plurality of initial probabilities. The initial distribution function is updated to form the new distribution function based on the received data value. The new distribution function includes a plurality of quantile points identifying the respective initial quantile estimates and a respective plurality of new probabilities associated with the respective initial quantile estimates. The approximation of the new distribution function is generated by, for each pair of adjacent quantile points in the new distribution function, connecting the adjacent quantile points using a linear approximation of a region between the adjacent quantile points. The new quantile estimates and the new probabilities associated with the new quantile estimates may then be stored.
摘要:
In one embodiment, a method of monitoring a network. The method includes: receiving, from each host of a set of two or more hosts of the network, a corresponding vector of M components constructed based on data packets received at the host during a time period, M being an integer greater than 1; and, based on the constructed vectors, using an expectation-maximization algorithm to estimate a cardinality distribution for the hosts in the set, wherein constructing a vector includes updating a component of the vector of the corresponding host in response to the corresponding host receiving a data packet, the updating including selecting the component for updating by hashing one or more fields of the data packet received by the corresponding host.
摘要:
A network-equipment-implemented method and apparatus for tracking durations of flows received at a network node in consecutive intervals utilizes two counting bloom filters in ping-pong operation to reduce memory and processing. Identifiers for flows that exceed a predetermined duration or number of intervals are stored in a long-duration flow-identifier table. Hash functions used within the counting bloom filters and optionally used in the long-duration flow-identifier table are chosen to minimize the probability of false positives in the detection of long-duration flows. In some embodiments, flows are sampled to conserve memory and processing resources at the risk of missing detection of some long-duration flows.
摘要:
Methods and apparatuses are provided for detecting traffic patterns in a data network. A sequential hashing scheme can be utilized that has D hash arrays. Each hash array i, wherein 1≦i≦D, includes Mi independent hash tables each having K buckets, with each of the buckets having an associated traffic total. Each of the keys corresponds with a single bucket of each of the Mi independent hash tables of each hash array i. The keys of the data network are partitioned into D words. As traffic is received for a key, a traffic total of each bucket that corresponds with a key is updated. The hash arrays can then be utilized to identify high traffic buckets of the independent hash tables having a traffic total greater than a threshold value. The high traffic buckets can be used to detect significant traffic patterns of the data network.
摘要:
In one embodiment, a method of monitoring a network. The method includes, at each node of a fixed set, constructing a corresponding vector of M components based on data packets received at the node during a time period, M being an integer greater than 1, the fixed set being formed of some nodes of the network; and, based on the constructed vectors, estimating how many of the received data packets have been received by all of the nodes of the set or estimating how many flows of the received data packets have data packets that have passed through all of the nodes of the set. The constructing includes updating a component of the vector of one of the nodes in response to the one of the nodes receiving a data packet. The updating includes selecting the component for updating by hashing a property of the data packet received by the one of the nodes.