-
公开(公告)号:US11829727B2
公开(公告)日:2023-11-28
申请号:US17239297
申请日:2021-04-23
Applicant: salesforce.com, inc.
Inventor: Jasdeep Singh , Nitish Shirish Keskar , Bryan McCann
Abstract: Approaches for cross-lingual regularization for multilingual generalization include a method for training a natural language processing (NLP) deep learning module. The method includes accessing a first dataset having a first training data entry, the first training data entry including one or more natural language input text strings in a first language; translating at least one of the one or more natural language input text strings of the first training data entry from the first language to a second language; creating a second training data entry by starting with the first training data entry and substituting the at least one of the natural language input text strings in the first language with the translation of the at least one of the natural language input text strings in the second language; adding the second training data entry to a second dataset; and training the deep learning module using the second dataset.
-
公开(公告)号:US11615249B2
公开(公告)日:2023-03-28
申请号:US16996726
申请日:2020-08-18
Applicant: salesforce.com, inc.
Inventor: Bryan McCann , Nitish Shirish Keskar , Caiming Xiong , Richard Socher
IPC: G06F40/30 , G06N3/08 , G06N5/04 , G06N3/04 , G06F40/56 , G06F16/242 , G06F16/33 , G06F16/332 , G06N20/20 , G06N20/10 , G06N20/00 , G10L15/16 , G10L15/18 , G06N3/044 , G06N3/045
Abstract: Approaches for multitask learning as question answering include an input layer for encoding a context and a question, a self-attention based transformer including an encoder and a decoder, a first bi-directional long-term short-term memory (biLSTM) for further encoding an output of the encoder, a long-term short-term memory (LSTM) for generating a context-adjusted hidden state from the output of the decoder and a hidden state, an attention network for generating first attention weights based on an output of the first biLSTM and an output of the LSTM, a vocabulary layer for generating a distribution over a vocabulary, a context layer for generating a distribution over the context, and a switch for generating a weighting between the distributions over the vocabulary and the context, generating a composite distribution based on the weighting, and selecting a word of an answer using the composite distribution.
-
公开(公告)号:US20220164635A1
公开(公告)日:2022-05-26
申请号:US17670368
申请日:2022-02-11
Applicant: salesforce.com, inc.
Inventor: Alexander Rosenberg Johansen , Bryan McCann , James Bradbury , Richard Socher
Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.
-
公开(公告)号:US20210249104A1
公开(公告)日:2021-08-12
申请号:US17001090
申请日:2020-08-24
Applicant: salesforce.com, inc.
Inventor: Ali Madani , Bryan McCann , Nikhil Naik
Abstract: The present disclosure provides systems and methods for controllable protein generation. According to some embodiments, the systems and methods leverage neural network models and techniques that have been developed for other fields, in particular, natural language processing (NLP). In some embodiments, the systems and methods use or employ models implemented with transformer architectures developed for language modeling and apply the same to generative modeling for protein engineering.
-
公开(公告)号:US20210150340A1
公开(公告)日:2021-05-20
申请号:US16877339
申请日:2020-05-18
Applicant: salesforce.com, inc.
Inventor: Wenhao Liu , Ka Chun Au , Shashank Harinath , Bryan McCann , Govardana Sachithanandam Ramachandran , Alexis Roos , Caiming Xiong
Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.
-
公开(公告)号:US20200285706A1
公开(公告)日:2020-09-10
申请号:US16399429
申请日:2019-04-30
Applicant: salesforce.com, inc.
Inventor: Jasdeep Singh , Nitish Shirish Keskar , Bryan McCann
Abstract: Approaches for cross-lingual regularization for multilingual generalization include a method for training a natural language processing (NLP) deep learning module. The method includes accessing a first dataset having a first training data entry, the first training data entry including one or more natural language input text strings in a first language; translating at least one of the one or more natural language input text strings of the first training data entry from the first language to a second language; creating a second training data entry by starting with the first training data entry and substituting the at least one of the natural language input text strings in the first language with the translation of the at least one of the natural language input text strings in the second language; adding the second training data entry to a second dataset; and training the deep learning module using the second dataset.
-
7.
公开(公告)号:US11769013B2
公开(公告)日:2023-09-26
申请号:US16680323
申请日:2019-11-11
Applicant: salesforce.com, inc.
Inventor: Michael Machado , James Douglas Harrison , Caiming Xiong , Xinyi Yang , Thomas Archie Cook , Roojuta Lalani , Jean-Marc Soumet , Karl Ryszard Skucha , Juan Rodriguez , Manju Vijayakumar , Vishal Motwani , Tian Xie , Bryan McCann , Nitish Shirish Keskar , Zhihao Zou , Chitra Gulabrani , Minal Khodani , Adarsha Badarinath , Rohiniben Thakar , Srikanth Kollu , Kevin Schoen , Qiong Liu , Amit Hetawal , Kevin Zhang , Kevin Zhang , Johnson Liu , Rafael Amsili
CPC classification number: G06F40/30 , G06F40/295 , G06N3/04 , G06N3/08 , H04L51/02
Abstract: A multi-tenant system performs custom configuration of a tenant-specific chatbot to process and act upon natural language requests. The multi-tenant system configures the tenant-specific chatbots without requiring tenant-specific training. The multi-tenant system providing a user interface for configuring a tenant-specific set of permitted actions. The multi-tenant system determines a set of example phrases for each of the selected permitted actions. The multi-tenant system receives a natural language request from a user and identifies the action that the user wants to perform. The multi-tenant system uses a neural network to compare the natural language request with example phrases to identify an example phrase that matches the natural language request. The multi-tenant system performs the action corresponding to the matching example phrase.
-
8.
公开(公告)号:US11657233B2
公开(公告)日:2023-05-23
申请号:US17673709
申请日:2022-02-16
Applicant: salesforce.com, inc.
Inventor: Nitish Shirish Keskar , Bryan McCann , Richard Socher , Caiming Xiong
IPC: G06F16/332 , G06F40/30 , G06F40/284 , G06N3/08
CPC classification number: G06F40/30 , G06F40/284 , G06F16/3329 , G06N3/08
Abstract: Systems and methods for unifying question answering and text classification via span extraction include a preprocessor for preparing a source text and an auxiliary text based on a task type of a natural language processing task, an encoder for receiving the source text and the auxiliary text from the preprocessor and generating an encoded representation of a combination of the source text and the auxiliary text, and a span-extractive decoder for receiving the encoded representation and identifying a span of text within the source text that is a result of the NLP task. The task type is one of entailment, classification, or regression. In some embodiments, the source text includes one or more of text received as input when the task type is entailment, a list of classifications when the task type is entailment or classification, or a list of similarity options when the task type is regression.
-
公开(公告)号:US11600194B2
公开(公告)日:2023-03-07
申请号:US16006691
申请日:2018-06-12
Applicant: salesforce.com, inc.
Inventor: Bryan McCann , Nitish Shirish Keskar , Caiming Xiong , Richard Socher
IPC: G09B7/02 , G06F16/9032 , G06F40/30 , G06F40/284 , G06N3/084 , G06F40/35 , G06N3/082 , G06N5/04 , G06N3/04 , G06F16/34 , G06F40/216
Abstract: Approaches for natural language processing include a multi-layer encoder for encoding words from a context and words from a question in parallel, a multi-layer decoder for decoding the encoded context and the encoded question, a pointer generator for generating distributions over the words from the context, the words from the question, and words in a vocabulary based on an output from the decoder, and a switch. The switch generates a weighting of the distributions over the words from the context, the words from the question, and the words in the vocabulary, generates a composite distribution based on the weighting of the distribution over the first words from the context, the distribution over the second words from the question, and the distribution over the words in the vocabulary, and selects words for inclusion in an answer using the composite distribution.
-
公开(公告)号:US11366969B2
公开(公告)日:2022-06-21
申请号:US16393801
申请日:2019-04-24
Applicant: salesforce.com, inc.
Inventor: Nazneen Rajani , Bryan McCann
IPC: G06F40/30 , G06F40/284 , G06N5/02
Abstract: According to some embodiments, systems and methods are provided to develop or provide common sense auto-generated explanations (CAGE) for the reasoning used by an artificial intelligence, neural network, or deep learning model to make a prediction. In some embodiments, the systems and methods use supervised fine-tuning on a language model (LM) to generate such explanations. These explanations may then be used for downstream classification.
-
-
-
-
-
-
-
-
-