公开(公告)号：US11922303B2

公开(公告)日：2024-03-05

申请号：US16877339

申请日：2020-05-18

申请人： Salesforce.com, Inc.

发明人： Wenhao Liu ,  Ka Chun Au ,  Shashank Harinath ,  Bryan McCann ,  Govardana Sachithanandam Ramachandran ,  Alexis Roos ,  Caiming Xiong

IPC分类号： G06N3/00 ,  G06F40/40 ,  G06N3/045 ,  G06N3/08

CPC分类号： G06N3/08 ,  G06F40/40 ,  G06N3/045

摘要： Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

公开(公告)号：US11775775B2

公开(公告)日：2023-10-03

申请号：US16695494

申请日：2019-11-26

申请人： salesforce.com, inc.

发明人： Akari Asai ,  Kazuma Hashimoto ,  Richard Socher ,  Caiming Xiong

IPC分类号： G06F40/40 ,  G06F40/30

CPC分类号： G06F40/40 ,  G06F40/30

摘要： Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.

公开(公告)号：US11741372B2

公开(公告)日：2023-08-29

申请号：US17397677

申请日：2021-08-09

申请人： salesforce.com, inc.

发明人： Lily Hu ,  Caiming Xiong ,  Richard Socher

IPC分类号： G06F16/906 ,  G06N3/088 ,  G06N3/08 ,  G06F18/21 ,  G06F18/2413 ,  G06V10/764 ,  G06V10/776 ,  G06V10/80 ,  G06V10/82 ,  G06F16/55

CPC分类号： G06N3/088 ,  G06F16/55 ,  G06F16/906 ,  G06F18/217 ,  G06F18/24137 ,  G06N3/08 ,  G06V10/764 ,  G06V10/776 ,  G06V10/811 ,  G06V10/82

摘要： Approaches to zero-shot learning include partitioning training data into first and second sets according to classes assigned to the training data, training a prediction module based on the first set to predict a cluster center based on a class label, training a correction module based on the second set and each of the class labels in the first set to generate a correction to a cluster center predicted by the prediction module, presenting a new class label for a new class to the prediction module to predict a new cluster center, presenting the new class label, the predicted new cluster center, and each of the class labels in the first set to the correction module to generate a correction for the predicted new cluster center, augmenting a classifier based on the corrected cluster center for the new class, and classifying input data into the new class using the classifier.

公开(公告)号：US11699297B2

公开(公告)日：2023-07-11

申请号：US17140987

申请日：2021-01-04

申请人： salesforce.com, inc.

发明人： Mingfei Gao ,  Zeyuan Chen ,  Le Xue ,  Ran Xu ,  Caiming Xiong

IPC分类号： G06V30/413 ,  G06F40/186 ,  G06F40/289 ,  G06V30/412 ,  G06F40/295 ,  G06V30/10 ,  G06V10/40

CPC分类号： G06V30/413 ,  G06F40/186 ,  G06F40/289 ,  G06V30/412 ,  G06F40/295 ,  G06V10/40 ,  G06V30/10

摘要： An online system extracts information from non-fixed form documents. The online system receives an image of a form document and obtains a set of phrases and locations of the set of phrases on the form image. For at least one field, the online system determines key scores for the set of phrases. The online system identifies a set of candidate values for the field from the set of identified phrases and identifies a set of neighbors for each candidate value from the set of identified phrases. The online system determines neighbor scores, where a neighbor score for a candidate value and a respective neighbor is determined based on the key score for the neighbor and a spatial relationship of the neighbor to the candidate value. The online system selects a candidate value and a respective neighbor based on the neighbor score as the value and key for the field.

公开(公告)号：US11676022B2

公开(公告)日：2023-06-13

申请号：US17460691

申请日：2021-08-30

申请人： salesforce.com, inc.

发明人： Ehsan Hosseini-Asl ,  Caiming Xiong ,  Yingbo Zhou ,  Richard Socher

IPC分类号： G05B13/02 ,  G10L21/003 ,  G10L15/07 ,  G10L15/065 ,  G06N3/02 ,  G06F18/21

CPC分类号： G05B13/027 ,  G06N3/02 ,  G10L21/003 ,  G06F18/2178 ,  G10L15/065 ,  G10L15/075

摘要： A method for training parameters of a first domain adaptation model. The method includes evaluating a cycle consistency objective using a first task specific model associated with a first domain and a second task specific model associated with a second domain, and evaluating one or more first discriminator models to generate a first discriminator objective using the second task specific model. The one or more first discriminator models include a plurality of discriminators corresponding to a plurality of bands that corresponds domain variable ranges of the first and second domains respectively. The method further includes updating, based on the cycle consistency objective and the first discriminator objective, one or more parameters of the first domain adaptation model for adapting representations from the first domain to the second domain.

公开(公告)号：US11657269B2

公开(公告)日：2023-05-23

申请号：US16592474

申请日：2019-10-03

申请人： salesforce.com, inc.

发明人： Tong Che ,  Caiming Xiong

IPC分类号： G06N3/08 ,  G06N3/04 ,  G06N20/20 ,  G06F17/18 ,  H03M7/30

CPC分类号： G06N3/08 ,  G06F17/18 ,  G06N3/0454 ,  G06N20/20 ,  H03M7/3059 ,  H03M7/6005 ,  H03M7/6011

摘要： Verification of discriminative models includes receiving an input; receiving a prediction from a discriminative model for the input; encoding, using an encoder, a latent variable based on the input; decoding, using a decoder, a reconstructed input based on the prediction and the latent variable; and determining, using an anomaly detection module, whether the prediction is reliable based on the input, the reconstructed input, and the latent variable. The encoder and the decoder are jointly trained to maximize an evidence lower bound of the encoder and the decoder. In some embodiments, the encoder and the decoder are further trained using a disentanglement constraint between the prediction and the latent variable. In some embodiments, the encoder and the decoder are further trained without using inputs that are out of a distribution of inputs used to train the discriminative model or that are adversarial to the discriminative model.

公开(公告)号：US20230120940A1

公开(公告)日：2023-04-20

申请号：US17589693

申请日：2022-01-31

申请人： salesforce.com, inc.

发明人： Liang Qiu ,  Chien-Sheng Wu ,  Wenhao Liu ,  Caiming Xiong

IPC分类号： G10L15/06 ,  G10L15/183 ,  G10L15/05

摘要： Embodiments described herein propose an approach for unsupervised structure extraction in task-oriented dialogues. Specifically, a Slot Boundary Detection (SBD) module is adopted, for which utterances from training domains are tagged with the conventional BIO schema but without the slot names. A transformer-based classifier is trained to detect the boundary of potential slot tokens in the test domain. Next, while the state number is usually unknown, it is more reasonable to assume the slot number is given when analyzing a dialogue system. The detected tokens are clustered into the number of slot of groups. Finally, the dialogue state is represented with a vector recording the modification times of every slot. The slot values are then tracked through each dialogue session in the corpus and label utterances with their dialogue states accordingly. The semantic structure is portrayed by computing the transition frequencies among the unique states.

公开(公告)号：US11620515B2

公开(公告)日：2023-04-04

申请号：US16716249

申请日：2019-12-16

申请人： salesforce.com, inc.

发明人： Linqing Liu ,  Caiming Xiong

IPC分类号： G06N3/08 ,  G06F40/40 ,  G06F40/30 ,  G06N3/04

摘要： Systems and methods are provided that employ knowledge distillation under a multi-task learning setting. In some embodiments, the systems and methods are implemented with a larger teacher model and a smaller student model, each of which comprise one or more shared layers and a plurality of task layers for performing multiple tasks. During training of the teacher model, its shared layers are initialized, and then the teacher model is multi-task refined. The teacher model predicts teacher logits. During training of the student model, its shared layers are initialized. Knowledge distillation is employed to transfer knowledge from the teacher model to the student model by the student model updating its shared layers and task layers, for example, according to the teacher logits of the teacher model. Other features are also provided.

公开(公告)号：US11605118B2

公开(公告)日：2023-03-14

申请号：US17112765

申请日：2020-12-04

申请人： salesforce.com, inc.

发明人： Yongjun Chen ,  Jia Li ,  Chenxi Li ,  Markus Anderle ,  Caiming Xiong ,  Simo Arajarvi ,  Harshavardhan Utharavalli

IPC分类号： G06Q30/00 ,  G06Q30/0601 ,  G06N3/08 ,  G06N3/04

摘要： Embodiments described herein provide an attentive network framework that models dynamic attributes with item and feature interactions. Specifically, the attentive network framework first encodes basket item sequences and dynamic attribute sequences with time-aware padding and time/month encoding to capture the seasonal patterns (e.g. in app recommendation, outdoor activities apps are more suitable for summer time while indoor activity apps are better for winter). Then the attentive network framework applies time-level attention modules on basket items' sequences and dynamic user attributes' sequences to capture basket items to basket items and attributes to attributes temporal sequential patterns. After that, an intra-basket attentive module is used on items in each basket to capture the correlation information among items.

公开(公告)号：US11544470B2

公开(公告)日：2023-01-03

申请号：US17005316

申请日：2020-08-28

申请人： salesforce.com, inc.

发明人： Jianguo Zhang ,  Kazuma Hashimoto ,  Chien-Sheng Wu ,  Wenhao Liu ,  Richard Socher ,  Caiming Xiong

IPC分类号： G06F17/00 ,  G06F40/30 ,  G10L15/16

摘要： An online system allows user interactions using natural language expressions. The online system uses a machine learning based model to infer an intent represented by a user expression. The machine learning based model takes as input a user expression and an example expression to compute a score indicating whether the user expression matches the example expression. Based on the scores, the intent inference module determines a most applicable intent for the expression. The online system determines a confidence threshold such that user expressions indicating a high confidence are assigned the most applicable intent and user expressions indicating a low confidence are assigned an out-of-scope intent. The online system encodes the example expressions using the machine learning based model. The online system may compare an encoded user expression with encoded example expressions to identify a subset of example expressions used to determine the most applicable intent.