-
公开(公告)号:US10929218B2
公开(公告)日:2021-02-23
申请号:US16400402
申请日:2019-05-01
Applicant: NEC Laboratories America, Inc.
Inventor: Jianwu Xu , Haifeng Chen
Abstract: A method for diagnosing computer system faults using log retrieval based on joint semantic and syntactic similarities includes receiving a set of query logs, defining joint semantic and syntactic similarities between the set of query logs and respective ones of multiple sets of historical logs based on semantic content and syntactic information obtained for the set of query logs and the multiple sets of historical logs, the multiple sets of historical logs being associated with historical computer system fault diagnoses, retrieving a set of historical logs from the multiple sets of historical logs to obtain a retrieved set of historical logs for computer system fault comparison based on a similarity measure corresponding to each of the multiple sets of historical logs derived from the joint semantic and syntactic similarities, and transmitting the retrieved set of historical logs to one or more computing devices to perform the computer system fault comparison.
-
公开(公告)号:US10706229B2
公开(公告)日:2020-07-07
申请号:US16145580
申请日:2018-09-28
Applicant: NEC Laboratories America, Inc.
Inventor: Jianwu Xu , Hui Zhang , Haifeng Chen , Tanay Kumar Saha
IPC: G06F17/40 , G06F40/211 , G06F7/06 , G06F40/30 , G06F40/216 , G06F40/284
Abstract: A computer-implemented method, system, and computer program product are provided for content aware heterogeneous log pattern comparative analysis. The method includes receiving, by a processor-device, a plurality of heterogeneous logs. The method also includes extracting, by the processor-device, a plurality of log syntactic patterns from the plurality of heterogenous logs. The method additionally includes generating, by the processor-device, latent representation vectors for each of the plurality of log syntactic patterns. The method further includes predicting, by the processor-device, an anomaly from the clustered latent representation vectors. The method also includes controlling an operation of a processor-based machine to react in accordance with the anomaly.
-
公开(公告)号:US20190354524A1
公开(公告)日:2019-11-21
申请号:US16400348
申请日:2019-05-01
Applicant: NEC Laboratories America, Inc.
Inventor: Jianwu Xu , Haifeng Chen
IPC: G06F16/2458 , G06F11/07 , G06F16/22
Abstract: Systems and methods for computer log retrieval are provided. A system can receive a set of query logs, and transform the set of query logs into a query log multi-variate time series. The system accesses log multivariate time series of historical logs, and computes and ranks a similarity distance between the query log multivariate time series and each of the log multivariate time series of the historical logs. The system also retrieves a highest ranked set of historical logs as a most similar set of logs compared to the set of query logs.
-
公开(公告)号:US10474642B2
公开(公告)日:2019-11-12
申请号:US15659131
申请日:2017-07-25
Applicant: NEC Laboratories America, Inc.
Inventor: Jianwu Xu , Biplob Debnath , Hui Zhang , Guofei Jiang
IPC: G06F17/00 , G06F16/178
Abstract: Methods and systems for log management include pre-processing heterogeneous logs and performing a log management action on the pre-processed plurality of heterogeneous logs. Pre-processing the logs includes performing a fixed tokenization of the heterogeneous logs based on a predefined set of symbols, performing a flexible tokenization of the heterogeneous logs based on a user-defined set of rules, converting timestamps in the heterogeneous logs to a single target timestamp format, and performing structural log tokenization of the heterogeneous logs based on user-defined structural information.
-
公开(公告)号:US20190340540A1
公开(公告)日:2019-11-07
申请号:US16400426
申请日:2019-05-01
Applicant: NEC Laboratories America, Inc.
Abstract: Systems and methods for adaptive and continuous log model learning can include updating a core model to generate an updated core model, each being a syntactic model and being additive in nature, based on a heterogeneous training log file and updating a peripheral model, that represents a relationship between core models, using a set of existing auxiliary files, that define can define relationship between existing models, and the updated core model to generate an updated peripheral model based on the heterogeneous training log file. Additionally, they can include detecting, with the updated core model and the updated peripheral model, an anomaly within a set of testing logs indicative of information technology system operation to take remedial action on the information technology system based on a most recent model update.
-
公开(公告)号:US20190171644A1
公开(公告)日:2019-06-06
申请号:US16207644
申请日:2018-12-03
Applicant: NEC Laboratories America, Inc.
Inventor: Jianwu Xu , Bo Zong , Haifeng Chen
IPC: G06F16/2453 , G06F16/2455 , G06F16/2458
Abstract: Methods and systems for event detection and correction include determining a log pattern for a received event. The log pattern is translated to an event search query. The event search query is weighted according to discriminative dimensions using term-frequency inverse-document-frequency. The event search query is matched to one or more known events. A corrective action is automatically performed based on a solution associated with the one or more known events.
-
公开(公告)号:US20180268312A1
公开(公告)日:2018-09-20
申请号:US15889666
申请日:2018-02-06
Applicant: NEC Laboratories America, Inc.
Inventor: Hui Zhang , Jianwu Xu , Biplob Debnath
CPC classification number: H04L63/1425 , G06N5/047 , G06N20/00 , H04L63/02 , H04L63/0209 , H04L63/0272 , H04L63/0281 , H04L63/1416 , H04L63/1458
Abstract: Systems and methods for enabling automated log analysis with controllable resource requirements are provided. A training set for log pattern learning is generated based on heterogeneous logs generated by a computer system. An incremental learning process is implemented to generate a set of log patterns from the training set. The heterogeneous logs are parsed using the set of log patterns. A set of applications is applied to the parsed logs.
-
公开(公告)号:US09928155B2
公开(公告)日:2018-03-27
申请号:US15352546
申请日:2016-11-15
Applicant: NEC Laboratories America, Inc.
Inventor: Jianwu Xu , Biplob Debnath , Hui Zhang , Guofei Jiang , Nipun Arora
CPC classification number: G06F11/3612 , G06F11/0706 , G06F11/0766 , G06F11/3636
Abstract: Systems and methods are disclosed for handling log data from one or more applications, sensors or instruments by receiving heterogeneous logs from arbitrary/unknown systems or applications; generating regular expression patterns from the heterogeneous log sources using machine learning and extracting a log pattern therefrom; generating models and profiles from training logs based on different conditions and updating a global model database storing all models generated over time; tokenizing raw log messages from one or more applications, sensors or instruments running a production system; transforming incoming tokenized streams are into data-objects for anomaly detection and forwarding of log messages to various anomaly detectors; and generating an anomaly alert from the one or more applications, sensors or instruments running a production system.
-
公开(公告)号:US20170277997A1
公开(公告)日:2017-09-28
申请号:US15430024
申请日:2017-02-10
Applicant: NEC Laboratories America, Inc.
Inventor: Bo Zong , Jianwu Xu , Guofei Jiang
CPC classification number: G06F16/2477 , G06F11/3072 , G06F16/35 , G06N5/045
Abstract: A method is provided that is performed in a network having nodes that generate heterogeneous logs including performance logs and text logs. The method includes performing, during a heterogeneous log training stage, (i) a log-to-time sequence conversion process for transforming clustered ones of training logs, from among the heterogeneous logs, into a set of time sequences that are each formed as a plurality of data pairs of a first configuration and a second configuration based on cluster type, (ii) a time series generation process for synchronizing particular ones of the time sequences in the set based on a set of criteria to output a set of fused time series, and (iii) an invariant model generation process for building invariant models for each time series data pair in the set of fused time series. The method includes controlling an anomaly-initiating one of the plurality of nodes based on the invariant models.
-
公开(公告)号:US20170236023A1
公开(公告)日:2017-08-17
申请号:US15351452
申请日:2016-11-15
Applicant: NEC Laboratories America, Inc.
Inventor: Biplob Debnath , Jianwu Xu , Hui Zhang , Guofei Jiang , Hossein Hamooni
CPC classification number: G06K9/4604 , G06F11/34 , G06F16/322 , G06F17/40
Abstract: Systems and methods are disclosed for parsing logs from arbitrary or unknown systems or applications by capturing heterogeneous logs from the arbitrary or unknown systems or applications; generating one pattern for every unique log message; building a pattern hierarchy tree by grouping patterns based on similarity metrics, and for every group it generates one pattern by combing all constituting patterns of that group; and selecting a set of patterns from the pattern hierarchy tree.
-
-
-
-
-
-
-
-
-