-
公开(公告)号：US20190340540A1
公开(公告)日：2019-11-07
申请号：US16400426
申请日：2019-05-01
Applicant: NEC Laboratories America, Inc.
Abstract: Systems and methods for adaptive and continuous log model learning can include updating a core model to generate an updated core model, each being a syntactic model and being additive in nature, based on a heterogeneous training log file and updating a peripheral model, that represents a relationship between core models, using a set of existing auxiliary files, that define can define relationship between existing models, and the updated core model to generate an updated peripheral model based on the heterogeneous training log file. Additionally, they can include detecting, with the updated core model and the updated peripheral model, an anomaly within a set of testing logs indicative of information technology system operation to take remedial action on the information technology system based on a most recent model update.
-
公开(公告)号：US10237295B2
公开(公告)日：2019-03-19
申请号：US15429849
申请日：2017-02-10
Applicant: NEC Laboratories America, Inc.
Inventor： Hui Zhang , Guofei Jiang
IPC: H04L29/06
Abstract: A system, program, and method for anomaly detection in heterogeneous logs. The system having a processor configured to identify pattern fields comprised of a plurality of event identifiers. The processor is further configured to generate an automata model by profiling event behaviors of the plurality of event sequences, the plurality of event sequences grouped in the automata model by combinations of one or more pattern fields and one or more event identifiers from among the plurality of event identifiers, wherein for a given combination, the one or more event identifiers therein must be respectively comprised in a same one of the one or more pattern fields with which it is combined. The processor is additionally configured to detect an anomaly in one of the plurality of event sequences using the automata model. The processor is also configured to control an anomaly-initiating one of the network devices based on the anomaly.
-
公开(公告)号：US20180365294A1
公开(公告)日：2018-12-20
申请号：US15983356
申请日：2018-05-18
Applicant: NEC Laboratories America, Inc.
Inventor： Daeki Cho , Nipun Arora , Hui Zhang
Abstract: Systems and methods for implementing a behavior analysis engine (BAE) to improve computer query processing are provided. A job request to execute an input rule on target log data is received by a BAE service via a user interface. The job request is executed by the BAE service to generate a result by obtaining the input rule from a rule-base, parsing the input rule to create a data structure, optimizing the data structure, and executing one or more operations using the optimized data structure. The result is stored by the BAE service in a result database.
-
公开(公告)号：US20180349250A1
公开(公告)日：2018-12-06
申请号：US15970398
申请日：2018-05-03
Applicant: NEC Laboratories America, Inc.
Inventor： Biplob Debnath , Hui Zhang , Erik Kruus
CPC classification number: G06F11/3476 , G06F11/0781 , G06F11/3072 , G06F11/3082 , G06F11/3608
Abstract: Systems and methods for implementing content-level anomaly detection for devices having limited memory are provided. At least one log content model is generated based on training log content of training logs obtained from one or more sources associated with the computer system. The at least one log content model is transformed into at least one modified log content model to limit memory usage. Anomaly detection is performed for testing log content of testing logs obtained from one or more sources associated with the computer system based on the at least one modified log content model. In response to the anomaly detection identifying one or more anomalies associated with the testing log content, the one or more anomalies are output.
-
35.发明授权
公开(公告)号：US10114728B2
公开(公告)日：2018-10-30
申请号：US14250340
申请日：2014-04-10
Applicant: NEC Laboratories America, Inc.
Inventor： Hui Zhang , Nipun Arora , Junghwan Rhee , Kai Ma , Guofei Jiang
IPC: G06F11/36
Abstract: The invention is directed to a computer implemented method and a system that implements an application performance profiler with hardware performance event information. The profiler provides dynamic tracing of application programs, and offers fine-grained hardware performance event profiling at function levels. To control the perturbation on target applications, the profiler also includes a control mechanism to constraint the function profiling overhead within a budget configured by users.
-
公开(公告)号：US20180268312A1
公开(公告)日：2018-09-20
申请号：US15889666
申请日：2018-02-06
Applicant: NEC Laboratories America, Inc.
Inventor： Hui Zhang , Jianwu Xu , Biplob Debnath
CPC classification number: H04L63/1425 , G06N5/047 , G06N20/00 , H04L63/02 , H04L63/0209 , H04L63/0272 , H04L63/0281 , H04L63/1416 , H04L63/1458
Abstract: Systems and methods for enabling automated log analysis with controllable resource requirements are provided. A training set for log pattern learning is generated based on heterogeneous logs generated by a computer system. An incremental learning process is implemented to generate a set of log patterns from the training set. The heterogeneous logs are parsed using the set of log patterns. A set of applications is applied to the parsed logs.
-
公开(公告)号：US09928155B2
公开(公告)日：2018-03-27
申请号：US15352546
申请日：2016-11-15
Applicant: NEC Laboratories America, Inc.
Inventor： Jianwu Xu , Biplob Debnath , Hui Zhang , Guofei Jiang , Nipun Arora
CPC classification number: G06F11/3612 , G06F11/0706 , G06F11/0766 , G06F11/3636
Abstract: Systems and methods are disclosed for handling log data from one or more applications, sensors or instruments by receiving heterogeneous logs from arbitrary/unknown systems or applications; generating regular expression patterns from the heterogeneous log sources using machine learning and extracting a log pattern therefrom; generating models and profiles from training logs based on different conditions and updating a global model database storing all models generated over time; tokenizing raw log messages from one or more applications, sensors or instruments running a production system; transforming incoming tokenized streams are into data-objects for anomaly detection and forwarding of log messages to various anomaly detectors; and generating an anomaly alert from the one or more applications, sensors or instruments running a production system.
-
公开(公告)号：US20170236023A1
公开(公告)日：2017-08-17
申请号：US15351452
申请日：2016-11-15
Applicant: NEC Laboratories America, Inc.
Inventor： Biplob Debnath , Jianwu Xu , Hui Zhang , Guofei Jiang , Hossein Hamooni
CPC classification number: G06K9/4604 , G06F11/34 , G06F16/322 , G06F17/40
Abstract: Systems and methods are disclosed for parsing logs from arbitrary or unknown systems or applications by capturing heterogeneous logs from the arbitrary or unknown systems or applications; generating one pattern for every unique log message; building a pattern hierarchy tree by grouping patterns based on similarity metrics, and for every group it generates one pattern by combing all constituting patterns of that group; and selecting a set of patterns from the pattern hierarchy tree.
-
公开(公告)号：US09489286B2
公开(公告)日：2016-11-08
申请号：US14168375
申请日：2014-01-30
Applicant: NEC Laboratories America, Inc.
Inventor： Nipun Arora , Hui Zhang , Junghwan Rhee , Guofei Jiang , Kenji Yoshihira
CPC classification number: G06F11/3644 , G06F11/3636
Abstract: This invention provides a new mechanism for “Hot-Tracing” using a novel placeholder mechanism and binary rewriting techniques, which leverages existing compiler flags in order to enable light-weight and highly flexible dynamic instrumentation. Broadly, I-Probe can be divided in 2 distinct workflows—1. Pre-processing (ColdPatch), and 2. Hot Tracing. The first phase is a pre-processing mechanism to prepare the binary for phase 2. The second phase is the actual hot-tracing mechanism, which allows users to dynamically instrument functions (more specifically symbols) of their choice.
Abstract translation: 本发明提供了一种使用新型占位符机制和二进制重写技术的“热追踪”的新机制，其利用现有的编译器标志以便实现轻量级和高度灵活的动态仪器。 普遍来说，I-Probe可以分为两个不同的工作流程 - 1。 预处理（ColdPatch）和2.热追踪。 第一阶段是为阶段2准备二进制的预处理机制。第二阶段是实际的热追踪机制，允许用户动态地对其选择的功能（更具体地说是符号）进行仪器仪表功能。
-
公开(公告)号：US20160063398A1
公开(公告)日：2016-03-03
申请号：US14839363
申请日：2015-08-28
Applicant: NEC Laboratories America, Inc.
Inventor： Hui Zhang , Xia Ning , Junghwan Rhee , Guofei Jiang , Hongteng Xu
CPC classification number: G06N99/005 , G06F9/5011 , G06F11/00 , G06F11/34 , G06F11/36 , G06N7/005
Abstract: A system and method for profiling a request in a service system with kernel events including a pre-processing module configured to obtain kernel event traces from the service system and determine starting and ending communication pairs of a request path for a request. A learning module is configured to learn pairwise relationships between the starting and ending communication pairs of training traces of sequential requests. A generation module is configured to generate communication paths for the request path from the starting and ending communication pairs of testing traces of concurrent requests using a heuristic procedure that is guided by the learned pairwise relationships and generate the request path for the request from the communication paths. The system and method precisely determine request paths for applications in a distributed system from kernel event traces even when there are numerous concurrent requests.
Abstract translation: 一种用于在具有内核事件的服务系统中对请求进行分析的系统和方法，所述内核事件包括预处理模块，所述预处理模块被配置为从所述服务系统获取内核事件跟踪并且确定请求的请求路径的起始和结束通信对。 学习模块被配置为学习顺序请求的训练轨迹的开始和结束通信对之间的成对关系。 生成模块被配置为使用由所学习的成对关系指导的启发式过程从并发请求的测试跟踪的起始和结束通信对生成针对请求路径的通信路径，并且从通信路径生成针对请求的请求路径 。 即使有许多并发请求，系统和方法也可以精确地确定来自内核事件跟踪的分布式系统中应用程序的请求路径。
