-
公开(公告)号:US10333805B2
公开(公告)日:2019-06-25
申请号:US15956392
申请日:2018-04-18
Applicant: NEC Laboratories America, Inc.
Inventor: Biplob Debnath , Hui Zhang
Abstract: A computer-implemented method for generating patterns from a set of heterogeneous log messages is presented. The method includes collecting the set of heterogenous log messages from arbitrary or unknown systems or applications or sensors or instruments, splitting the log messages into tokens based on a set of delimiters, identifying datatypes of the tokens, identifying a log structure of the log messages by generating pattern-signatures of all the tokens and the datatypes based on predefined pattern settings, generating a pattern for each of the log structures and enabling users to edit the pattern for each of the log structures based on user requirements.
-
公开(公告)号:US20180307576A1
公开(公告)日:2018-10-25
申请号:US15956381
申请日:2018-04-18
Applicant: NEC Laboratories America, Inc.
Inventor: Biplob Debnath , Hui Zhang
CPC classification number: G06F11/34 , G06F11/0775 , G06F11/079 , G06F11/3072 , G06F16/35 , G06F2201/86
Abstract: A system and method are provided for pattern discovery in input heterogeneous logs having unstructured text content and one or more fields. The system includes a memory. The system further includes a processor in communication with the memory. The processor runs program code to preprocess the input heterogeneous logs to obtain pre-processed logs by splitting the input heterogeneous logs into tokens. The processor runs program code to generate seed patterns from the preprocessed logs. The processor runs program code to generate final patterns by specializing a selected set of fields in each of the seed patterns to generate a final pattern set.
-
公开(公告)号:US20180165147A1
公开(公告)日:2018-06-14
申请号:US15830579
申请日:2017-12-04
Applicant: NEC Laboratories America, Inc.
Inventor: Biplob Debnath , Hui Zhang , Guofei Jiang
Abstract: A computer-implemented method, computer program product, and computer processing system are provided. The method includes preprocessing, by a processor, a set of heterogeneous logs by splitting each of the logs into tokens to obtain preprocessed logs. Each of the logs in the set is associated with a timestamp and textual content in one or more fields. The method further includes generating, by the processor, a set of regular expressions from the preprocessed logs. The method also includes performing, by the processor, an unsupervised parsing operation by applying the regular expressions to the preprocessed logs to obtain a set of parsed logs and a set of unparsed logs, if any. The method additionally includes storing, by the processor, the set of parsed logs in a log analytics database and the set of unparsed logs in a debugging database.
-
公开(公告)号:US20180137001A1
公开(公告)日:2018-05-17
申请号:US15810960
申请日:2017-11-13
Applicant: NEC Laboratories America, Inc.
Inventor: Bo Zong , LuAn Tang , Qi Song , Biplob Debnath , Hui Zhang , Guofei Jiang
CPC classification number: G06F11/079 , G06F11/0793 , G06F11/3476 , G06N3/084 , G06N5/022 , G06N5/04
Abstract: A method is provided that includes transforming training data into a neural network based learning model using a set of temporal graphs derived from the training data. The method includes performing model learning on the learning model by automatically adjusting learning model parameters based on the set of the temporal graphs to minimize differences between a predetermined ground-truth ranking list and a learning model output ranking list. The method includes transforming testing data into a neural network based inference model using another set of temporal graphs derived from the testing data. The method includes performing model inference by applying the inference and learning models to test data to extract context features for alerts in the test data and calculate a ranking list for the alerts based on the extracted context features. Top-ranked alerts are identified as critical alerts. Each alert represents an anomaly in the test data.
-
公开(公告)号:US20180129579A1
公开(公告)日:2018-05-10
申请号:US15784393
申请日:2017-10-16
Applicant: NEC Laboratories America, Inc.
Inventor: Biplob Debnath , Nipun Arora , Hui Zhang , Guofei Jiang , Mohiuddin Solaimani , Muhammad Ali Gulzar
CPC classification number: G06F11/3476 , G06F11/0706 , G06F11/0775 , G06F11/0787 , G06F11/3065 , G06N20/00
Abstract: Systems and methods are disclosed for processing a stream of logged data by: creating one or more models from a set of training logs during a training phase; receiving testing data in real-time and generating anomalies using the models created during the training phase; updating the one or more models during real-time processing of a live stream of logs; and detecting a log anomaly from the live stream of logs.
-
公开(公告)号:US20180060748A1
公开(公告)日:2018-03-01
申请号:US15684293
申请日:2017-08-23
Applicant: NEC Laboratories America, Inc.
Inventor: Jianwu Xu , Biplob Debnath , Bo Zong , Hui Zhang , Guofei Jiang , Hancheng Ge
CPC classification number: G06N5/047 , G06F17/16 , G06F17/20 , G06F17/2282 , G06F17/277 , G06N7/00
Abstract: A heterogeneous log pattern editing recommendation system and computer-implemented method are provided. The system has a processor configured to identify, from heterogeneous logs, patterns including variable fields and constant fields. The processor is also configured to extract a category feature, a cardinality feature, and a before-after n-gram feature by tokenizing the variable fields in the identified patterns. The processor is additionally configured to generate target similarity scores between target fields to be potentially edited and other fields from among the variable fields in the heterogeneous logs using pattern editing operations based on the extracted category feature, the extracted cardinality feature, and the extracted before-after n-gram feature. The processor is further configured to recommend, to a user, log pattern edits for at least one of the target fields based on the target similarity scores between the target fields in the heterogeneous logs.
-
-
-
-
-