-
公开(公告)号:US11615249B2
公开(公告)日:2023-03-28
申请号:US16996726
申请日:2020-08-18
Applicant: salesforce.com, inc.
Inventor: Bryan McCann , Nitish Shirish Keskar , Caiming Xiong , Richard Socher
IPC: G06F40/30 , G06N3/08 , G06N5/04 , G06N3/04 , G06F40/56 , G06F16/242 , G06F16/33 , G06F16/332 , G06N20/20 , G06N20/10 , G06N20/00 , G10L15/16 , G10L15/18 , G06N3/044 , G06N3/045
Abstract: Approaches for multitask learning as question answering include an input layer for encoding a context and a question, a self-attention based transformer including an encoder and a decoder, a first bi-directional long-term short-term memory (biLSTM) for further encoding an output of the encoder, a long-term short-term memory (LSTM) for generating a context-adjusted hidden state from the output of the decoder and a hidden state, an attention network for generating first attention weights based on an output of the first biLSTM and an output of the LSTM, a vocabulary layer for generating a distribution over a vocabulary, a context layer for generating a distribution over the context, and a switch for generating a weighting between the distributions over the vocabulary and the context, generating a composite distribution based on the weighting, and selecting a word of an answer using the composite distribution.
-
公开(公告)号:US11537801B2
公开(公告)日:2022-12-27
申请号:US17214691
申请日:2021-03-26
Applicant: salesforce.com, inc.
Inventor: Kazuma Hashimoto , Raffaella Buschiazzo , James Bradbury , Teresa Marshall , Caiming Xiong , Richard Socher
Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.
-
公开(公告)号:US11514915B2
公开(公告)日:2022-11-29
申请号:US16175639
申请日:2018-10-30
Applicant: salesforce.com, inc.
Inventor: Chien-Sheng Wu , Caiming Xiong , Richard Socher
IPC: G10L15/00 , G10L15/28 , G10L15/22 , G06N5/00 , G06F16/335 , G06F16/332 , G06F16/33
Abstract: A system and corresponding method are provided for generating responses for a dialogue between a user and a computer. The system includes a memory storing information for a dialogue history and a knowledge base. An encoder may receive a new utterance from the user and generate a global memory pointer used for filtering the knowledge base information in the memory. A decoder may generate at least one local memory pointer and a sketch response for the new utterance. The sketch response includes at least one sketch tag to be replaced by knowledge base information from the memory. The system generates the dialogue computer response using the local memory pointer to select a word from the filtered knowledge base information to replace the at least one sketch tag in the sketch response.
-
公开(公告)号:US11347708B2
公开(公告)日:2022-05-31
申请号:US16680302
申请日:2019-11-11
Applicant: salesforce.com, inc.
Inventor: Ankit Chadha , Zeyuan Chen , Caiming Xiong , Ran Xu , Richard Socher
Abstract: Embodiments described herein provide unsupervised density-based clustering to infer table structure from document. Specifically, a number of words are identified from a block of text in an noneditable document, and the spatial coordinates of each word relative to the rectangular region are identified. Based on the word density of the rectangular region, the words are grouped into clusters using a heuristic radius search method. Words that are grouped into the same cluster are determined to be the element that belong to the same cell. In this way, the cells of the table structure can be identified. Once the cells are identified based on the word density of the block of text, the identified cells can be expanded horizontally or grouped vertically to identify rows or columns of the table structure.
-
公开(公告)号:US20220164635A1
公开(公告)日:2022-05-26
申请号:US17670368
申请日:2022-02-11
Applicant: salesforce.com, inc.
Inventor: Alexander Rosenberg Johansen , Bryan McCann , James Bradbury , Richard Socher
Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.
-
86.
公开(公告)号:US20220083837A1
公开(公告)日:2022-03-17
申请号:US17534298
申请日:2021-11-23
Applicant: salesforce.com, inc.
Inventor: Kazuma Hashimoto , Caiming Xiong , Richard Socher
IPC: G06N3/04 , G06N3/08 , G06F40/30 , G06F40/205 , G06F40/216 , G06F40/253 , G06F40/284 , G06N3/063
Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.
-
公开(公告)号:US11227218B2
公开(公告)日:2022-01-18
申请号:US15980207
申请日:2018-05-15
Applicant: salesforce.com, inc.
Inventor: Sewon Min , Victor Zhong , Caiming Xiong , Richard Socher
Abstract: A natural language processing system that includes a sentence selector and a question answering module. The sentence selector receives a question and sentences that are associated with a context. For a question and each sentence, the sentence selector determines a score. A score represents whether the question is answerable with the sentence. Sentence selector then generates a minimum set of sentences from the scores associated with the question and sentences. The question answering module generates an answer for the question from the minimum set of sentences.
-
公开(公告)号:US20210397799A1
公开(公告)日:2021-12-23
申请号:US17463227
申请日:2021-08-31
Applicant: salesforce.com, inc.
Inventor: Kazuma Hashimoto , Raffaella Buschiazzo , James Bradbury , Teresa Anna Marshall , Caiming Xiong , Richard Socher
IPC: G06F40/58
Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.
-
公开(公告)号:US11113598B2
公开(公告)日:2021-09-07
申请号:US15221532
申请日:2016-07-27
Applicant: salesforce.com, inc.
Inventor: Richard Socher , Ankit Kumar , Ozan Irsoy , Mohit Iyyer , Caiming Xiong , Stephen Merity , Romain Paulus
IPC: G06N3/08 , G06N3/04 , G06F16/33 , G06F16/332
Abstract: A novel unified neural network framework, the dynamic memory network, is disclosed. This unified framework reduces every task in natural language processing to a question answering problem over an input sequence. Inputs and questions are used to create and connect deep memory sequences. Answers are then generated based on dynamically retrieved memories.
-
公开(公告)号:US11106182B2
公开(公告)日:2021-08-31
申请号:US16054935
申请日:2018-08-03
Applicant: salesforce.com, inc.
Inventor: Ehsan Hosseini-Asl , Caiming Xiong , Yingbo Zhou , Richard Socher
IPC: G05B13/02 , G06N3/02 , G10L21/003 , G10L15/065 , G10L15/07 , G06K9/62
Abstract: A method for training parameters of a first domain adaptation model includes evaluating a cycle consistency objective using a first task specific model associated with a first domain and a second task specific model associated with a second domain. The evaluating the cycle consistency objective is based on one or more first training representations adapted from the first domain to the second domain by a first domain adaptation model and from the second domain to the first domain by a second domain adaptation model, and one or more second training representations adapted from the second domain to the first domain by the second domain adaptation model and from the first domain to the second domain by the first domain adaptation model. The method further includes evaluating a learning objective based on the cycle consistency objective, and updating parameters of the first domain adaptation model based on learning objective.
-
-
-
-
-
-
-
-
-