公开(公告)号：US12198047B2

公开(公告)日：2025-01-14

申请号：US17122894

申请日：2020-12-15

Applicant: Salesforce.com, inc.

Inventor： James Bradbury ,  Stephen Joseph Merity ,  Caiming Xiong ,  Richard Socher

IPC: G06N3/08 ,  G06F17/16 ,  G06F40/00 ,  G06F40/216 ,  G06F40/30 ,  G06F40/44 ,  G06N3/04 ,  G06N3/044 ,  G06N3/045 ,  G06N3/10 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30

Abstract: The technology disclosed provides a quasi-recurrent neural network (QRNN) encoder-decoder model that alternates convolutional layers, which apply in parallel across timesteps, and minimalist recurrent pooling layers that apply in parallel across feature dimensions.

公开(公告)号：US12164878B2

公开(公告)日：2024-12-10

申请号：US17581380

申请日：2022-01-21

Applicant: Salesforce.com, Inc.

Inventor： Tong Niu ,  Kazuma Hashimoto ,  Yingbo Zhou ,  Caiming Xiong

IPC: G06F40/51

Abstract: Embodiments described herein provide a cross-lingual sentence alignment framework that is trained only on rich-resource language pairs. To obtain an accurate aligner, a pretrained multi-lingual language model is used, and a classifier is trained on parallel data from rich-resource language pairs. This trained classifier may then be used for cross-lingual transfer with low-resource languages.

公开(公告)号：US11922303B2

公开(公告)日：2024-03-05

申请号：US16877339

申请日：2020-05-18

Applicant: Salesforce.com, Inc.

Inventor： Wenhao Liu ,  Ka Chun Au ,  Shashank Harinath ,  Bryan McCann ,  Govardana Sachithanandam Ramachandran ,  Alexis Roos ,  Caiming Xiong

IPC: G06N3/00 ,  G06F40/40 ,  G06N3/045 ,  G06N3/08

CPC classification number: G06N3/08 ,  G06F40/40 ,  G06N3/045

Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

公开(公告)号：US11775775B2

公开(公告)日：2023-10-03

申请号：US16695494

申请日：2019-11-26

Applicant: salesforce.com, inc.

Inventor： Akari Asai ,  Kazuma Hashimoto ,  Richard Socher ,  Caiming Xiong

IPC: G06F40/40 ,  G06F40/30

CPC classification number: G06F40/40 ,  G06F40/30

Abstract: Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.

公开(公告)号：US11741372B2

公开(公告)日：2023-08-29

申请号：US17397677

申请日：2021-08-09

Applicant: salesforce.com, inc.

Inventor： Lily Hu ,  Caiming Xiong ,  Richard Socher

IPC: G06F16/906 ,  G06N3/088 ,  G06N3/08 ,  G06F18/21 ,  G06F18/2413 ,  G06V10/764 ,  G06V10/776 ,  G06V10/80 ,  G06V10/82 ,  G06F16/55

CPC classification number: G06N3/088 ,  G06F16/55 ,  G06F16/906 ,  G06F18/217 ,  G06F18/24137 ,  G06N3/08 ,  G06V10/764 ,  G06V10/776 ,  G06V10/811 ,  G06V10/82

Abstract: Approaches to zero-shot learning include partitioning training data into first and second sets according to classes assigned to the training data, training a prediction module based on the first set to predict a cluster center based on a class label, training a correction module based on the second set and each of the class labels in the first set to generate a correction to a cluster center predicted by the prediction module, presenting a new class label for a new class to the prediction module to predict a new cluster center, presenting the new class label, the predicted new cluster center, and each of the class labels in the first set to the correction module to generate a correction for the predicted new cluster center, augmenting a classifier based on the corrected cluster center for the new class, and classifying input data into the new class using the classifier.

公开(公告)号：US11699297B2

公开(公告)日：2023-07-11

申请号：US17140987

申请日：2021-01-04

Applicant: salesforce.com, inc.

Inventor： Mingfei Gao ,  Zeyuan Chen ,  Le Xue ,  Ran Xu ,  Caiming Xiong

IPC: G06V30/413 ,  G06F40/186 ,  G06F40/289 ,  G06V30/412 ,  G06F40/295 ,  G06V30/10 ,  G06V10/40

CPC classification number: G06V30/413 ,  G06F40/186 ,  G06F40/289 ,  G06V30/412 ,  G06F40/295 ,  G06V10/40 ,  G06V30/10

Abstract: An online system extracts information from non-fixed form documents. The online system receives an image of a form document and obtains a set of phrases and locations of the set of phrases on the form image. For at least one field, the online system determines key scores for the set of phrases. The online system identifies a set of candidate values for the field from the set of identified phrases and identifies a set of neighbors for each candidate value from the set of identified phrases. The online system determines neighbor scores, where a neighbor score for a candidate value and a respective neighbor is determined based on the key score for the neighbor and a spatial relationship of the neighbor to the candidate value. The online system selects a candidate value and a respective neighbor based on the neighbor score as the value and key for the field.

公开(公告)号：US11676022B2

公开(公告)日：2023-06-13

申请号：US17460691

申请日：2021-08-30

Applicant: salesforce.com, inc.

Inventor： Ehsan Hosseini-Asl ,  Caiming Xiong ,  Yingbo Zhou ,  Richard Socher

IPC: G05B13/02 ,  G10L21/003 ,  G10L15/07 ,  G10L15/065 ,  G06N3/02 ,  G06F18/21

CPC classification number: G05B13/027 ,  G06N3/02 ,  G10L21/003 ,  G06F18/2178 ,  G10L15/065 ,  G10L15/075

Abstract: A method for training parameters of a first domain adaptation model. The method includes evaluating a cycle consistency objective using a first task specific model associated with a first domain and a second task specific model associated with a second domain, and evaluating one or more first discriminator models to generate a first discriminator objective using the second task specific model. The one or more first discriminator models include a plurality of discriminators corresponding to a plurality of bands that corresponds domain variable ranges of the first and second domains respectively. The method further includes updating, based on the cycle consistency objective and the first discriminator objective, one or more parameters of the first domain adaptation model for adapting representations from the first domain to the second domain.

公开(公告)号：US11657269B2

公开(公告)日：2023-05-23

申请号：US16592474

申请日：2019-10-03

Applicant: salesforce.com, inc.

Inventor： Tong Che ,  Caiming Xiong

IPC: G06N3/08 ,  G06N3/04 ,  G06N20/20 ,  G06F17/18 ,  H03M7/30

CPC classification number: G06N3/08 ,  G06F17/18 ,  G06N3/0454 ,  G06N20/20 ,  H03M7/3059 ,  H03M7/6005 ,  H03M7/6011

Abstract: Verification of discriminative models includes receiving an input; receiving a prediction from a discriminative model for the input; encoding, using an encoder, a latent variable based on the input; decoding, using a decoder, a reconstructed input based on the prediction and the latent variable; and determining, using an anomaly detection module, whether the prediction is reliable based on the input, the reconstructed input, and the latent variable. The encoder and the decoder are jointly trained to maximize an evidence lower bound of the encoder and the decoder. In some embodiments, the encoder and the decoder are further trained using a disentanglement constraint between the prediction and the latent variable. In some embodiments, the encoder and the decoder are further trained without using inputs that are out of a distribution of inputs used to train the discriminative model or that are adversarial to the discriminative model.

公开(公告)号：US20230120940A1

公开(公告)日：2023-04-20

申请号：US17589693

申请日：2022-01-31

Applicant: salesforce.com, inc.

Inventor： Liang Qiu ,  Chien-Sheng Wu ,  Wenhao Liu ,  Caiming Xiong

IPC: G10L15/06 ,  G10L15/183 ,  G10L15/05

Abstract: Embodiments described herein propose an approach for unsupervised structure extraction in task-oriented dialogues. Specifically, a Slot Boundary Detection (SBD) module is adopted, for which utterances from training domains are tagged with the conventional BIO schema but without the slot names. A transformer-based classifier is trained to detect the boundary of potential slot tokens in the test domain. Next, while the state number is usually unknown, it is more reasonable to assume the slot number is given when analyzing a dialogue system. The detected tokens are clustered into the number of slot of groups. Finally, the dialogue state is represented with a vector recording the modification times of every slot. The slot values are then tracked through each dialogue session in the corpus and label utterances with their dialogue states accordingly. The semantic structure is portrayed by computing the transition frequencies among the unique states.

公开(公告)号：US11620515B2

公开(公告)日：2023-04-04

申请号：US16716249

申请日：2019-12-16

Applicant: salesforce.com, inc.

Inventor： Linqing Liu ,  Caiming Xiong

IPC: G06N3/08 ,  G06F40/40 ,  G06F40/30 ,  G06N3/04

Abstract: Systems and methods are provided that employ knowledge distillation under a multi-task learning setting. In some embodiments, the systems and methods are implemented with a larger teacher model and a smaller student model, each of which comprise one or more shared layers and a plurality of task layers for performing multiple tasks. During training of the teacher model, its shared layers are initialized, and then the teacher model is multi-task refined. The teacher model predicts teacher logits. During training of the student model, its shared layers are initialized. Knowledge distillation is employed to transfer knowledge from the teacher model to the student model by the student model updating its shared layers and task layers, for example, according to the teacher logits of the teacher model. Other features are also provided.