公开(公告)号：US20190251431A1

公开(公告)日：2019-08-15

申请号：US15974075

申请日：2018-05-08

Applicant: salesforce.com, inc.

Inventor： Nitish Shirish Keskar ,  Bryan McCann ,  Caiming Xiong ,  Richard Socher

IPC: G06N3/08 ,  G06F17/28

CPC classification number: G06N3/08 ,  G06F17/2785 ,  G06F17/2881 ,  G06N3/0445 ,  G06N3/0454 ,  G06N5/04

Abstract: Approaches for multitask learning as question answering include a method for training that includes receiving a plurality of training samples including training samples from a plurality of task types, presenting the training samples to a neural model to generate an answer, determining an error between the generated answer and the natural language ground truth answer for each training sample presented, and adjusting parameters of the neural model based on the error. Each of the training samples includes a natural language context, question, and ground truth answer. An order in which the training samples are presented to the neural model includes initially selecting the training samples according to a first training strategy and switching to selecting the training samples according to a second training strategy. In some embodiments the first training strategy is a sequential training strategy and the second training strategy is a joint training strategy.

公开(公告)号：US20180373682A1

公开(公告)日：2018-12-27

申请号：US15982841

申请日：2018-05-17

Applicant: salesforce.com, inc.

Inventor： Bryan McCann ,  Caiming Xiong ,  Richard Socher

IPC: G06F17/22 ,  G06F17/28 ,  G06N3/08 ,  G06F17/27

Abstract: A system is provided for natural language processing. In some embodiments, the system includes an encoder for generating context-specific word vectors for at least one input sequence of words. The encoder is pre-trained using training data for performing a first natural language processing task. A neural network performs a second natural language processing task on the at least one input sequence of words using the context-specific word vectors. The first natural language process task is different from the second natural language processing task and the neural network is separately trained from the encoder. In some embodiments, the first natural processing task can be machine translation, and the second natural processing task can be one of sentiment analysis, question classification, entailment classification, and question answering

公开(公告)号：US11501076B2

公开(公告)日：2022-11-15

申请号：US15974075

申请日：2018-05-08

Applicant: salesforce.com, inc.

Inventor： Nitish Shirish Keskar ,  Bryan McCann ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/30 ,  G06N3/08 ,  G06N5/04 ,  G06N3/04 ,  G06F40/56 ,  G06F16/242 ,  G06F16/33 ,  G06F16/332 ,  G06N20/20 ,  G06N20/10 ,  G06N20/00

Abstract: Approaches for multitask learning as question answering include a method for training that includes receiving a plurality of training samples including training samples from a plurality of task types, presenting the training samples to a neural model to generate an answer, determining an error between the generated answer and the natural language ground truth answer for each training sample presented, and adjusting parameters of the neural model based on the error. Each of the training samples includes a natural language context, question, and ground truth answer. An order in which the training samples are presented to the neural model includes initially selecting the training samples according to a first training strategy and switching to selecting the training samples according to a second training strategy. In some embodiments the first training strategy is a sequential training strategy and the second training strategy is a joint training strategy.

公开(公告)号：US11281863B2

公开(公告)日：2022-03-22

申请号：US16518905

申请日：2019-07-22

Applicant: salesforce.com, inc.

Inventor： Nitish Shirish Keskar ,  Bryan McCann ,  Richard Socher ,  Caiming Xiong

IPC: G06F16/332 ,  G06F40/30 ,  G06F40/284 ,  G06N3/08

Abstract: Systems and methods for unifying question answering and text classification via span extraction include a preprocessor for preparing a source text and an auxiliary text based on a task type of a natural language processing task, an encoder for receiving the source text and the auxiliary text from the preprocessor and generating an encoded representation of a combination of the source text and the auxiliary text, and a span-extractive decoder for receiving the encoded representation and identifying a span of text within the source text that is a result of the NLP task. The task type is one of entailment, classification, or regression. In some embodiments, the source text includes one or more of text received as input when the task type is entailment, a list of classifications when the task type is entailment or classification, or a list of similarity options when the task type is regression.

公开(公告)号：US20220058348A1

公开(公告)日：2022-02-24

申请号：US17124317

申请日：2020-12-16

Applicant: salesforce.com, inc.

Inventor： Tianxing He ,  Ehsan Hosseini-Asl ,  Bryan McCann ,  Caiming Xiong

IPC: G06F40/58

Abstract: Embodiments described herein provide natural language processing (NLP) systems and methods that utilize energy-based models (EBMs) to compute an exponentially-weighted energy-like term in the loss function to train an NLP classifier. Specifically, noise contrastive estimation (NCE) procedures are applied together with the EBM-based loss objectives for training the NLPs.

公开(公告)号：US20210249105A1

公开(公告)日：2021-08-12

申请号：US17001045

申请日：2020-08-24

Applicant: salesforce.com, inc.

Inventor： Ali Madani ,  Bryan McCann ,  Nikhil Naik

IPC: G16B40/30 ,  G16B25/10 ,  G16B50/10

Abstract: The present disclosure provides systems and methods for controllable protein generation. According to some embodiments, the systems and methods leverage neural network models and techniques that have been developed for other fields, in particular, natural language processing (NLP). In some embodiments, the systems and methods use or employ models implemented with transformer architectures developed for language modeling and apply the same to generative modeling for protein engineering.

公开(公告)号：US20210249100A1

公开(公告)日：2021-08-12

申请号：US17001068

申请日：2020-08-24

Applicant: salesforce.com, inc.

Inventor： Ali Madani ,  Bryan McCann ,  Nikhil Naik

IPC: G16B5/20 ,  G16B30/00 ,  G16B15/20

Abstract: The present disclosure provides systems and methods for controllable protein generation. According to some embodiments, the systems and methods leverage neural network models and techniques that have been developed for other fields, in particular, natural language processing (NLP). In some embodiments, the systems and methods use or employ models implemented with transformer architectures developed for language modeling and apply the same to generative modeling for protein engineering.

公开(公告)号：US20210124876A1

公开(公告)日：2021-04-29

申请号：US16750598

申请日：2020-01-23

Applicant: salesforce.com, inc.

Inventor： Wojciech Kryscinski ,  Bryan McCann

IPC: G06F40/30 ,  G06F40/268 ,  G06F16/34 ,  G06K9/62

Abstract: A weakly-supervised, model-based approach is provided for verifying or checking factual consistency and identifying conflicts between source documents and a generated summary. In some embodiments, an artificially generated training dataset is created by applying rule-based transformations to sentences sampled from one or more unannotated source documents of a dataset. Each of the resulting transformed sentences can be either semantically variant or invariant from the respective original sampled sentence, and labeled accordingly. In some embodiments, the generated training dataset is used to train a factual consistency checking model. The factual consistency checking model can classify whether a corresponding text summary is factually consistent with a source text document, and if so, may identify a span in the source text document that supports the corresponding text summary.

公开(公告)号：US10776581B2

公开(公告)日：2020-09-15

申请号：US15974118

申请日：2018-05-08

Applicant: salesforce.com, inc.

Inventor： Bryan McCann ,  Nitish Shirish Keskar ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/30 ,  G06N3/08 ,  G06N5/04 ,  G06N3/04 ,  G06F40/56 ,  G06F16/242 ,  G06F16/33 ,  G06F16/332

Abstract: Approaches for multitask learning as question answering include an input layer for encoding a context and a question, a self-attention based transformer including an encoder and a decoder, a first bi-directional long-term short-term memory (biLSTM) for further encoding an output of the encoder, a long-term short-term memory (LSTM) for generating a context-adjusted hidden state from the output of the decoder and a hidden state, an attention network for generating first attention weights based on an output of the first biLSTM and an output of the LSTM, a vocabulary layer for generating a distribution over a vocabulary, a context layer for generating a distribution over the context, and a switch for generating a weighting between the distributions over the vocabulary and the context, generating a composite distribution based on the weighting, and selecting a word of an answer using the composite distribution.

公开(公告)号：US20190355270A1

公开(公告)日：2019-11-21

申请号：US16006691

申请日：2018-06-12

Applicant: salesforce.com, inc.

Inventor： Bryan McCann ,  Nitish Shirish Keskar ,  Caiming Xiong ,  Richard Socher

IPC: G09B7/02

Abstract: Approaches for natural language processing include a multi-layer encoder for encoding words from a context and words from a question in parallel, a multi-layer decoder for decoding the encoded context and the encoded question, a pointer generator for generating distributions over the words from the context, the words from the question, and words in a vocabulary based on an output from the decoder, and a switch. The switch generates a weighting of the distributions over the words from the context, the words from the question, and the words in the vocabulary, generates a composite distribution based on the weighting of the distribution over the first words from the context, the distribution over the second words from the question, and the distribution over the words in the vocabulary, and selects words for inclusion in an answer using the composite distribution.