Ultra-fast pattern generation algorithm for the heterogeneous logs

    公开(公告)号:US10333805B2

    公开(公告)日:2019-06-25

    申请号:US15956392

    申请日:2018-04-18

    Abstract: A computer-implemented method for generating patterns from a set of heterogeneous log messages is presented. The method includes collecting the set of heterogenous log messages from arbitrary or unknown systems or applications or sensors or instruments, splitting the log messages into tokens based on a set of delimiters, identifying datatypes of the tokens, identifying a log structure of the log messages by generating pattern-signatures of all the tokens and the datatypes based on predefined pattern settings, generating a pattern for each of the log structures and enabling users to edit the pattern for each of the log structures based on user requirements.

    FIELD CONTENT BASED PATTERN GENERATION FOR HETEROGENEOUS LOGS

    公开(公告)号:US20180307576A1

    公开(公告)日:2018-10-25

    申请号:US15956381

    申请日:2018-04-18

    Abstract: A system and method are provided for pattern discovery in input heterogeneous logs having unstructured text content and one or more fields. The system includes a memory. The system further includes a processor in communication with the memory. The processor runs program code to preprocess the input heterogeneous logs to obtain pre-processed logs by splitting the input heterogeneous logs into tokens. The processor runs program code to generate seed patterns from the preprocessed logs. The processor runs program code to generate final patterns by specializing a selected set of fields in each of the seed patterns to generate a final pattern set.

    STRUCTURE-LEVEL ANOMALY DETECTION FOR UNSTRUCTURED LOGS

    公开(公告)号:US20180165147A1

    公开(公告)日:2018-06-14

    申请号:US15830579

    申请日:2017-12-04

    Abstract: A computer-implemented method, computer program product, and computer processing system are provided. The method includes preprocessing, by a processor, a set of heterogeneous logs by splitting each of the logs into tokens to obtain preprocessed logs. Each of the logs in the set is associated with a timestamp and textual content in one or more fields. The method further includes generating, by the processor, a set of regular expressions from the preprocessed logs. The method also includes performing, by the processor, an unsupervised parsing operation by applying the regular expressions to the preprocessed logs to obtain a set of parsed logs and a set of unparsed logs, if any. The method additionally includes storing, by the processor, the set of parsed logs in a log analytics database and the set of unparsed logs in a debugging database.

    RECOMMENDER SYSTEM FOR HETEROGENEOUS LOG PATTERN EDITING OPERATION

    公开(公告)号:US20180060748A1

    公开(公告)日:2018-03-01

    申请号:US15684293

    申请日:2017-08-23

    Abstract: A heterogeneous log pattern editing recommendation system and computer-implemented method are provided. The system has a processor configured to identify, from heterogeneous logs, patterns including variable fields and constant fields. The processor is also configured to extract a category feature, a cardinality feature, and a before-after n-gram feature by tokenizing the variable fields in the identified patterns. The processor is additionally configured to generate target similarity scores between target fields to be potentially edited and other fields from among the variable fields in the heterogeneous logs using pattern editing operations based on the extracted category feature, the extracted cardinality feature, and the extracted before-after n-gram feature. The processor is further configured to recommend, to a user, log pattern edits for at least one of the target fields based on the target similarity scores between the target fields in the heterogeneous logs.

Patent Agency Ranking