公开(公告)号：US20160162456A1

公开(公告)日：2016-06-09

申请号：US14964517

申请日：2015-12-09

申请人： Robert J. Munro ,  Schuyler D. Erle ,  Christopher Walker ,  Sarah K. Luger ,  Jason Brenier ,  Gary C. King ,  Paul A. Tepper ,  Ross Mechanic ,  Andrew Gilchrist-Scott ,  Jessica D. Long ,  James B. Robinson ,  Brendan D. Callahan ,  Michelle Casbon ,  Ujjwal Sarin ,  Aneesh Nair ,  Veena Basavaraj ,  Tripti Saxena ,  Edgar Nunez ,  Martha G. Hinrichs ,  Haley Most ,  Tyler J. Schnoebelen

发明人： Robert J. Munro ,  Schuyler D. Erle ,  Christopher Walker ,  Sarah K. Luger ,  Jason Brenier ,  Gary C. King ,  Paul A. Tepper ,  Ross Mechanic ,  Andrew Gilchrist-Scott ,  Jessica D. Long ,  James B. Robinson ,  Brendan D. Callahan ,  Michelle Casbon ,  Ujjwal Sarin ,  Aneesh Nair ,  Veena Basavaraj ,  Tripti Saxena ,  Edgar Nunez ,  Martha G. Hinrichs ,  Haley Most ,  Tyler J. Schnoebelen

IPC分类号： G06F17/24 ,  G06F17/22 ,  G06F17/28

CPC分类号： G06F17/30598 ,  G06F3/0482 ,  G06F17/2241 ,  G06F17/241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06F17/30011 ,  G06F17/30401 ,  G06F17/30445 ,  G06F17/30604 ,  G06F17/30654 ,  G06F17/30705 ,  G06F17/30734 ,  G06F17/30864 ,  G06Q50/01

摘要： Methods are presented for generating a natural language model. The method may comprise: ingesting training data representative of documents to be analyzed by the natural language model, generating a hierarchical data structure comprising at least two topical nodes within which the training data is to be subdivided into by the natural language model, selecting a plurality of documents among the training data to be annotated, generating an annotation prompt for each document configured to elicit an annotation about said document indicating which node among the at least two topical nodes said document is to be classified into, receiving the annotation based on the annotation prompt; and generating the natural language model using an adaptive machine learning process configured to determine patterns among the annotations for how the documents in the training data are to be subdivided according to the at least two topical nodes of the hierarchical data structure.

摘要翻译： 提出了生成自然语言模型的方法。 该方法可以包括：摄取表示要由自然语言模型分析的文档的训练数据，生成包括至少两个主题节点的分层数据结构，训练数据将在该节点内被自然语言模型细分，选择多个 在要注释的训练数据中生成文档的注释提示，为每个文档生成关于所述文档的注释的注释提示，该注释指示所述文档中的至少两个主题节点中的哪个节点被分类，基于注释接收注释 提示; 以及使用自适应机器学习过程来生成所述自然语言模型，所述自适应机器学习过程被配置为根据所述分级数据结构的所述至少两个主题节点来确定所述注释中的模式如何根据所述训练数据中的文档被细分。

公开(公告)号：US20160162458A1

公开(公告)日：2016-06-09

申请号：US14964522

申请日：2015-12-09

申请人： Robert J. Munro ,  Christopher Walker ,  Sarah K. Luger ,  Jason Brenier ,  Paul A. Tepper ,  Ross Mechanic ,  Andrew Gilchrist-Scott ,  Gary C. King ,  Brendan D. Callahan ,  Tyler J. Schnoebelen ,  Edgar Nunez ,  Haley Most

发明人： Robert J. Munro ,  Christopher Walker ,  Sarah K. Luger ,  Jason Brenier ,  Paul A. Tepper ,  Ross Mechanic ,  Andrew Gilchrist-Scott ,  Gary C. King ,  Brendan D. Callahan ,  Tyler J. Schnoebelen ,  Edgar Nunez ,  Haley Most

IPC分类号： G06F17/24 ,  G06F3/0482

CPC分类号： G06F17/241 ,  G06F3/0482 ,  G06F16/243 ,  G06F16/24532 ,  G06F16/285 ,  G06F16/288 ,  G06F16/3329 ,  G06F16/35 ,  G06F16/367 ,  G06F16/93 ,  G06F16/951 ,  G06F17/2241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06Q50/01

摘要： Methods and systems are disclosed for creating and linking a series of interfaces configured to display information and receive confirmation of classifications made by a natural language modeling engine to improve organization of a collection of documents into an hierarchical structure. In some embodiments, the interfaces may display to an annotator a plurality of labels of potential classifications for a document as identified by a natural language modeling engine, collect annotated responses from the annotator, aggregate the annotated responses across other annotators, analyze the accuracy of the natural language modeling engine based on the aggregated annotated responses, and predict accuracies of the natural language modeling engine's classifications of the documents.

摘要翻译： 公开了用于创建和链接被配置为显示信息并且接收由自然语言建模引擎进行的分类的确认的一系列接口以改进文档集合到分层结构中的组合的方法和系统。 在一些实施例中，接口可以向注释器显示由自然语言建模引擎识别的文档的潜在分类的多个标签，从注释器收集注释的响应，在其他注释器之间聚合注释的响应，分析 基于聚合注释响应的自然语言建模引擎，并预测自然语言建模引擎对文档分类的准确性。

公开(公告)号：US20160162457A1

公开(公告)日：2016-06-09

申请号：US14964520

申请日：2015-12-09

申请人： Robert J. Munro ,  Schuyler D. Erle ,  Jason Brenier ,  Paul A. Tepper ,  Tripti Saxena ,  Gary C. King ,  Jessica D. Long ,  Brendan D. Callahan ,  Tyler J. Schnoebelen ,  Stefan Krawczyk ,  Veena Basavaraj

发明人： Robert J. Munro ,  Schuyler D. Erle ,  Jason Brenier ,  Paul A. Tepper ,  Tripti Saxena ,  Gary C. King ,  Jessica D. Long ,  Brendan D. Callahan ,  Tyler J. Schnoebelen ,  Stefan Krawczyk ,  Veena Basavaraj

IPC分类号： G06F17/24 ,  G06F17/28

CPC分类号： G06F17/241 ,  G06F3/0482 ,  G06F16/243 ,  G06F16/24532 ,  G06F16/285 ,  G06F16/288 ,  G06F16/3329 ,  G06F16/35 ,  G06F16/367 ,  G06F16/93 ,  G06F16/951 ,  G06F17/2241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06Q50/01

摘要： Methods, apparatuses and computer readable medium are presented for generating a natural language model. A method for generating a natural language model comprises: selecting from a pool of documents, a first set of documents to be annotated; receiving annotations of the first set of documents elicited by first human readable prompts; training a natural language model using the annotated first set of documents; determining documents in the pool having uncertain natural language processing results according to the trained natural language model and/or the received annotations; selecting from the pool of documents, a second set of documents to be annotated comprising documents having uncertain natural language processing results; receiving annotations of the second set of documents elicited by second human readable prompts; and retraining a natural language model using the annotated second set of documents.

摘要翻译： 提出了用于生成自然语言模型的方法，装置和计算机可读介质。 一种用于生成自然语言模型的方法包括：从文档池中选择要注释的第一组文档; 接收由第一可读提示引起的第一组文档的注释; 使用注释的第一组文件训练自然语言模型; 根据训练的自然语言模型和/或接收的注释，确定具有不确定的自然语言处理结果的池中的文档; 从文件池中选择要注释的第二组文档，其中包括具有不确定的自然语言处理结果的文档; 接收由第二可读提示引起的第二组文件的注释; 并使用注释的第二组文档重新培训自然语言模型。

公开(公告)号：US20190243886A1

公开(公告)日：2019-08-08

申请号：US16125343

申请日：2018-09-07

申请人： Schuyler D. Erle ,  Robert J. Munro ,  Brendan D. Callahan ,  Jason Brenier ,  Paul A. Tepper ,  Jessica D. Long ,  James B. Robinson ,  Aneesh Nair ,  Michelle Casbon ,  Stefan Krawczyk

发明人： Schuyler D. Erle ,  Robert J. Munro ,  Brendan D. Callahan ,  Jason Brenier ,  Paul A. Tepper ,  Jessica D. Long ,  James B. Robinson ,  Aneesh Nair ,  Michelle Casbon ,  Stefan Krawczyk

IPC分类号： G06F17/24 ,  G06F16/332 ,  G06F17/28 ,  G06F16/28 ,  G06F16/93 ,  G06F16/35 ,  G06F16/2453 ,  G06F16/951 ,  G06F16/242 ,  G06F17/22 ,  G06Q50/00 ,  G06F17/27 ,  G06F3/0482 ,  G06F16/36

CPC分类号： G06F17/241 ,  G06F3/0482 ,  G06F16/243 ,  G06F16/24532 ,  G06F16/285 ,  G06F16/288 ,  G06F16/3329 ,  G06F16/35 ,  G06F16/367 ,  G06F16/93 ,  G06F16/951 ,  G06F17/2241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06Q50/01

摘要： Systems and methods are presented for providing improved machine performance in natural language processing. In some example embodiments, an API module is presented that is configured to drive processing of a system architecture for natural language processing. Aspects of the present disclosure allow for a natural language model to classify documents while other documents are being retrieved in real time. The natural language model and the documents are configured to be stored in a stateless format, which also allows for additional functions to be performed on the documents while the natural language model is used to continue classifying other documents.

公开(公告)号：US20160162569A1

公开(公告)日：2016-06-09

申请号：US14964510

申请日：2015-12-09

申请人： Schuyler D. Erle ,  Robert J. Munro ,  Brendan D. Callahan ,  Jason Brenier ,  Paul A. Tepper ,  Jessica D. Long ,  James B. Robinson ,  Aneesh Nair ,  Michelle Casbon ,  Stefan Krawczyk

发明人： Schuyler D. Erle ,  Robert J. Munro ,  Brendan D. Callahan ,  Jason Brenier ,  Paul A. Tepper ,  Jessica D. Long ,  James B. Robinson ,  Aneesh Nair ,  Michelle Casbon ,  Stefan Krawczyk

IPC分类号： G06F17/30 ,  G06F17/27 ,  G06F17/24

CPC分类号： G06F17/241 ,  G06F3/0482 ,  G06F16/243 ,  G06F16/24532 ,  G06F16/285 ,  G06F16/288 ,  G06F16/3329 ,  G06F16/35 ,  G06F16/367 ,  G06F16/93 ,  G06F16/951 ,  G06F17/2241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06Q50/01

摘要： Systems and methods are presented for providing improved machine performance in natural language processing. In some example embodiments, an API module is presented that is configured to drive processing of a system architecture for natural language processing. Aspects of the present disclosure allow for a natural language model to classify documents while other documents are being retrieved in real time. The natural language model and the documents are configured to be stored in a stateless format, which also allows for additional functions to be performed on the documents while the natural language model is used to continue classifying other documents.

摘要翻译： 提出了系统和方法，用于在自然语言处理中提供改进的机器性能。 在一些示例性实施例中，呈现了被配置为驱动用于自然语言处理的系统架构的处理的API模块。 本公开的方面允许自然语言模型对文档进行分类，而实时检索其他文档。 自然语言模型和文档被配置为以无状态格式存储，这也允许在文档上执行附加功能，同时使用自然语言模型继续对其他文档进行分类。

公开(公告)号：US20160162464A1

公开(公告)日：2016-06-09

申请号：US14964528

申请日：2015-12-09

申请人： Robert J. Munro ,  Christopher Walker ,  Sarah K. Luger ,  Brendan D. Callahan ,  Gary C. King ,  Paul A. Tepper ,  Jana N. Thompson ,  Tyler J. Schnoebelen ,  Jason Brenier ,  Jessica D. Long

发明人： Robert J. Munro ,  Christopher Walker ,  Sarah K. Luger ,  Brendan D. Callahan ,  Gary C. King ,  Paul A. Tepper ,  Jana N. Thompson ,  Tyler J. Schnoebelen ,  Jason Brenier ,  Jessica D. Long

IPC分类号： G06F17/27

CPC分类号： G06F17/241 ,  G06F3/0482 ,  G06F16/243 ,  G06F16/24532 ,  G06F16/285 ,  G06F16/288 ,  G06F16/3329 ,  G06F16/35 ,  G06F16/367 ,  G06F16/93 ,  G06F16/951 ,  G06F17/2241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06Q50/01

摘要： Methods, apparatuses and computer readable medium are presented for generating a natural language model. A method for generating a natural language model comprises: receiving more than one annotation of a document; calculating a level of agreement among the received annotations; determining that a criterion among a first criterion, a second criterion, and a third criterion is satisfied based at least in part on the level of agreement; determining an aggregated annotation representing an aggregation of information in the received annotations and training a natural language model using the aggregated annotation, when the first criterion is satisfied; generating at least one human readable prompt configured to receive additional annotations of the document, when the second criterion is satisfied; and discarding the received annotations from use in training the natural language model, when the third criterion is satisfied.

摘要翻译： 提出了用于生成自然语言模型的方法，装置和计算机可读介质。 一种用于生成自然语言模型的方法包括：接收多于一个文档的注释; 计算接收到的注释之间的一致程度; 确定至少部分地基于协议水平来满足第一标准，第二标准和第三标准中的标准; 当满足所述第一标准时，确定表示所接收的注释中的信息的聚合的聚合注释并训练使用所述聚合注释的自然语言模型; 当满足所述第二标准时，生成至少一个人可读提示，其被配置为接收所述文档的附加注释; 并且当满足第三标准时，丢弃所接收的注释用于训练自然语言模型。

公开(公告)号：US09495345B2

公开(公告)日：2016-11-15

申请号：US14964511

申请日：2015-12-09

申请人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler J. Schnoebelen ,  Jason Brenier ,  Jessica D. Long ,  Brendan D. Callahan ,  Paul A. Tepper ,  Edgar Nunez

发明人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler J. Schnoebelen ,  Jason Brenier ,  Jessica D. Long ,  Brendan D. Callahan ,  Paul A. Tepper ,  Edgar Nunez

IPC分类号： G06F17/27 ,  G06F17/21 ,  G06F17/24 ,  G06F17/28 ,  G06F17/22 ,  G06F3/0482 ,  G06F17/30

CPC分类号： G06F17/30598 ,  G06F3/0482 ,  G06F17/2241 ,  G06F17/241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06F17/30011 ,  G06F17/30401 ,  G06F17/30445 ,  G06F17/30604 ,  G06F17/30654 ,  G06F17/30705 ,  G06F17/30734 ,  G06F17/30864 ,  G06Q50/01

摘要： Systems and methods are presented for the automatic placement of rules applied to topics in a logical hierarchy when conducting natural language processing. In some embodiments, a method includes: accessing, at a child node in a logical hierarchy, at least one rule associated with the child node; identifying a percolation criterion associated with a parent node to the child node, said percolation criterion indicating that the at least one rule associated with the child node is to be associated also with the parent node; associating the at least one rule with the parent node such that the at least one rule defines a second factor for determining whether the document is to also be classified into the parent node; accessing the document for natural language processing; and determining whether the document is to be classified into the parent node or the child node based on the at least one rule.

摘要翻译： 提出了系统和方法，用于在进行自然语言处理时自动放置适用于逻辑层次结构中主题的规则。 在一些实施例中，一种方法包括：在逻辑层级中的子节点处访问与所述子节点相关联的至少一个规则; 识别与所述子节点的父节点相关联的渗透标准，所述渗透标准指示与所述子节点相关联的所述至少一个规则也将与所述父节点相关联; 将所述至少一个规则与所述父节点相关联，使得所述至少一个规则定义用于确定所述文档是否也被分类为所述父节点的第二因素; 访问文档进行自然语言处理; 以及基于所述至少一个规则来确定所述文档是否被分类到所述父节点或所述子节点。

公开(公告)号：US20160162476A1

公开(公告)日：2016-06-09

申请号：US14964511

申请日：2015-12-09

申请人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler J. Schnoebelen ,  Jason Brenier ,  Jessica D. Long ,  Brendan D. Callahan ,  Paul A. Tepper ,  Edgar Nunez

发明人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler J. Schnoebelen ,  Jason Brenier ,  Jessica D. Long ,  Brendan D. Callahan ,  Paul A. Tepper ,  Edgar Nunez

IPC分类号： G06F17/28

CPC分类号： G06F17/30598 ,  G06F3/0482 ,  G06F17/2241 ,  G06F17/241 ,  G06F17/272 ,  G06F17/2785 ,  G06F17/28 ,  G06F17/2809 ,  G06F17/30011 ,  G06F17/30401 ,  G06F17/30445 ,  G06F17/30604 ,  G06F17/30654 ,  G06F17/30705 ,  G06F17/30734 ,  G06F17/30864 ,  G06Q50/01

摘要： Systems and methods are presented for the automatic placement of rules applied to topics in a logical hierarchy when conducting natural language processing. In some embodiments, a method includes: accessing, at a child node in a logical hierarchy, at least one rule associated with the child node; identifying a percolation criterion associated with a parent node to the child node, said percolation criterion indicating that the at least one rule associated with the child node is to be associated also with the parent node; associating the at least one rule with the parent node such that the at least one rule defines a second factor for determining whether the document is to also be classified into the parent node; accessing the document for natural language processing; and determining whether the document is to be classified into the parent node or the child node based on the at least one rule.

摘要翻译： 提出了系统和方法，用于在进行自然语言处理时自动放置适用于逻辑层次结构中主题的规则。 在一些实施例中，一种方法包括：在逻辑层级中的子节点处访问与所述子节点相关联的至少一个规则; 识别与所述子节点的父节点相关联的渗透标准，所述渗透标准指示与所述子节点相关联的所述至少一个规则也将与所述父节点相关联; 将所述至少一个规则与所述父节点相关联，使得所述至少一个规则定义用于确定所述文档是否也被分类为所述父节点的第二因素; 访问文档进行自然语言处理; 以及基于所述至少一个规则来确定所述文档是否被分类到所述父节点或所述子节点。

公开(公告)号：US20160162467A1

公开(公告)日：2016-06-09

申请号：US14964525

申请日：2015-12-09

申请人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler J. Schnoebelen ,  Brendan D. Callahan ,  Jessica D. Long ,  Gary C. King ,  Paul A. Tepper ,  Jason Brenier ,  Stefan Krawczyk

发明人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler J. Schnoebelen ,  Brendan D. Callahan ,  Jessica D. Long ,  Gary C. King ,  Paul A. Tepper ,  Jason Brenier ,  Stefan Krawczyk

IPC分类号： G06F17/27

CPC分类号： G06F17/277 ,  G06F17/2715 ,  G06F17/2785

摘要： Methods, apparatuses, and systems are presented for generating natural language models using a novel system architecture for feature extraction. A method for extracting features for natural language processing comprises: accessing one or more tokens generated from a document to be processed; receiving one or more feature types defined by user; receiving selection of one or more feature types from a plurality of system-defined and user-defined feature types, wherein each feature type comprises one or more rules for generating features; receiving one or more parameters for the selected feature types, wherein the one or more rules for generating features are defined at least in part by the parameters; generating features associated with the document to be processed based on the selected feature types and the received parameters; and outputting the generated features in a format common among all feature types.

摘要翻译： 提出了使用用于特征提取的新型系统架构来生成自然语言模型的方法，装置和系统。 一种用于提取自然语言处理特征的方法，包括：访问从要处理的文档生成的一个或多个令牌; 接收用户定义的一个或多个特征类型; 从多个系统定义和用户定义的特征类型中接收对一个或多个特征类型的选择，其中每个特征类型包括用于生成特征的一个或多个规则; 为所选择的特征类型接收一个或多个参数，其中用于生成特征的所述一个或多个规则至少部分地由所述参数定义; 基于所选择的特征类型和接收到的参数来生成与要处理的文档相关联的特征; 并以所有特征类型中共同的格式输出生成的特征。

公开(公告)号：US20180157636A1

公开(公告)日：2018-06-07

申请号：US15814349

申请日：2017-11-15

申请人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler j. Schnoebelen ,  Brendan D. Callahan ,  Jessica D. Long ,  Gary C. King ,  Paul A. Tepper ,  Jason A. Brenier ,  Stefan Krawczyk

发明人： Robert J. Munro ,  Schuyler D. Erle ,  Tyler j. Schnoebelen ,  Brendan D. Callahan ,  Jessica D. Long ,  Gary C. King ,  Paul A. Tepper ,  Jason A. Brenier ,  Stefan Krawczyk

IPC分类号： G06F17/27

CPC分类号： G06F17/277 ,  G06F17/2715 ,  G06F17/2785

摘要： Methods, apparatuses, and systems are presented for generating natural language models using a novel system architecture for feature extraction. A method for extracting features for natural language processing comprises: accessing one or more tokens generated from a document to be processed; receiving one or more feature types defined by user; receiving selection of one or more feature types from a plurality of system-defined and user-defined feature types, wherein each feature type comprises one or more rules for generating features; receiving one or more parameters for the selected feature types, wherein the one or more rules for generating features are defined at least in part by the parameters; generating features associated with the document to be processed based on the selected feature types and the received parameters; and outputting the generated features in a format common among all feature types.