公开(公告)号：US20220083837A1

公开(公告)日：2022-03-17

申请号：US17534298

申请日：2021-11-23

Applicant: salesforce.com, inc.

Inventor： Kazuma Hashimoto ,  Caiming Xiong ,  Richard Socher

IPC: G06N3/04 ,  G06N3/08 ,  G06F40/30 ,  G06F40/205 ,  G06F40/216 ,  G06F40/253 ,  G06F40/284 ,  G06N3/063

Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.

公开(公告)号：US20210397799A1

公开(公告)日：2021-12-23

申请号：US17463227

申请日：2021-08-31

Applicant: salesforce.com, inc.

Inventor： Kazuma Hashimoto ,  Raffaella Buschiazzo ,  James Bradbury ,  Teresa Anna Marshall ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/58

Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.

公开(公告)号：US12164878B2

公开(公告)日：2024-12-10

申请号：US17581380

申请日：2022-01-21

Applicant: Salesforce.com, Inc.

Inventor： Tong Niu ,  Kazuma Hashimoto ,  Yingbo Zhou ,  Caiming Xiong

IPC: G06F40/51

Abstract: Embodiments described herein provide a cross-lingual sentence alignment framework that is trained only on rich-resource language pairs. To obtain an accurate aligner, a pretrained multi-lingual language model is used, and a classifier is trained on parallel data from rich-resource language pairs. This trained classifier may then be used for cross-lingual transfer with low-resource languages.

公开(公告)号：US11775775B2

公开(公告)日：2023-10-03

申请号：US16695494

申请日：2019-11-26

Applicant: salesforce.com, inc.

Inventor： Akari Asai ,  Kazuma Hashimoto ,  Richard Socher ,  Caiming Xiong

IPC: G06F40/40 ,  G06F40/30

CPC classification number: G06F40/40 ,  G06F40/30

Abstract: Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.

公开(公告)号：US20230059870A1

公开(公告)日：2023-02-23

申请号：US17565305

申请日：2021-12-29

Applicant: salesforce.com, inc.

Inventor： Xi Ye ,  Semih Yavuz ,  Kazuma Hashimoto ,  Yingbo Zhou

IPC: G06N5/04 ,  G06V30/19

Abstract: Embodiments described herein provide a question answering approach that answers a question by generating an executable logical form. First, a ranking model is used to select a set of good logical forms from a pool of logical forms obtained by searching over a knowledge graph. The selected logical forms are good in the sense that they are close to (or exactly match, in some cases) the intents in the question and final desired logical form. Next, a generation model is adopted conditioned on the question as well as the selected logical forms to generate the target logical form and execute it to obtain the final answer. For example, at inference stage, when a question is received, a matching logical form is identified from the question, based on which the final answer can be generated based on the node that is associated with the matching logical form in the knowledge base.

公开(公告)号：US20230054068A1

公开(公告)日：2023-02-23

申请号：US17589522

申请日：2022-01-31

Applicant: salesforce.com, inc.

Inventor： Haopeng Zheng ,  Semih Yavuz ,  Wojciech Kryscinski ,  Kazuma Hashimoto ,  Yingbo Zhou

IPC: G06F40/166 ,  G06F40/279 ,  G06F40/117 ,  G06N20/00

Abstract: Embodiments described herein provide document summarization systems and methods that utilize fine-tuning of pre-trained abstractive summarization models to produce summaries that more faithfully track the content of the documents. Such abstractive summarization models may be pre-trained using a corpus consisting of pairs of articles and associated summaries. For each article-summary pair, a pseudo label or control code is generated and represents a faithfulness of the summary with respect to the article. The pre-trained model is then fine-tuned based on the article-summary pairs and the corresponding control codes. The resulting fine-tuned models then provide improved faithfulness in document summarization tasks.

公开(公告)号：US11544470B2

公开(公告)日：2023-01-03

申请号：US17005316

申请日：2020-08-28

Applicant: salesforce.com, inc.

Inventor： Jianguo Zhang ,  Kazuma Hashimoto ,  Chien-Sheng Wu ,  Wenhao Liu ,  Richard Socher ,  Caiming Xiong

IPC: G06F17/00 ,  G06F40/30 ,  G10L15/16

Abstract: An online system allows user interactions using natural language expressions. The online system uses a machine learning based model to infer an intent represented by a user expression. The machine learning based model takes as input a user expression and an example expression to compute a score indicating whether the user expression matches the example expression. Based on the scores, the intent inference module determines a most applicable intent for the expression. The online system determines a confidence threshold such that user expressions indicating a high confidence are assigned the most applicable intent and user expressions indicating a low confidence are assigned an out-of-scope intent. The online system encodes the example expressions using the machine learning based model. The online system may compare an encoded user expression with encoded example expressions to identify a subset of example expressions used to determine the most applicable intent.

公开(公告)号：US20220101844A1

公开(公告)日：2022-03-31

申请号：US17037556

申请日：2020-09-29

Applicant: salesforce.com, inc.

Inventor： Xinyi Yang ,  Tian Xie ,  Caiming Xiong ,  Wenhao Liu ,  Huan Wang ,  Kazuma Hashimoto ,  Yingbo Zhou ,  Xugang Ye ,  Jin Qu ,  Feihong Wu

IPC: G10L15/22 ,  G10L15/26 ,  G10L15/16 ,  G10L15/30

Abstract: A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.

公开(公告)号：US20220036884A1

公开(公告)日：2022-02-03

申请号：US17500855

申请日：2021-10-13

Applicant: salesforce.com, inc.

Inventor： Govardana Sachithanandam Ramachandran ,  Kazuma Hashimoto ,  Caiming Xiong ,  Richard Socher

IPC: G10L15/16 ,  G10L15/06 ,  G06N3/04 ,  G06N3/08 ,  G06K9/62

Abstract: Embodiments described herein provide safe policy improvement (SPI) in a batch reinforcement learning framework for a task-oriented dialogue. Specifically, a batch reinforcement learning framework for dialogue policy learning is provided, which improves the performance of the dialogue and learns to shape a reward that reasons the invention behind human response rather than just imitating the human demonstration.

公开(公告)号：US20210375269A1

公开(公告)日：2021-12-02

申请号：US16999426

申请日：2020-08-21

Applicant: salesforce.com, inc.

Inventor： Semih Yavuz ,  Kazuma Hashimoto ,  Wenhao Liu ,  Nitish Shirish Keskar ,  Richard Socher ,  Caiming Xiong

IPC: G10L15/183 ,  G06N20/00 ,  G10L15/06 ,  G06F17/18

Abstract: Embodiments described herein utilize pre-trained masked language models as the backbone for dialogue act tagging and provide cross-domain generalization of the resulting dialogue acting taggers. For example, a pre-trained MASK token of BERT model may be used as a controllable mechanism for augmenting text input, e.g., generating tags for an input of unlabeled dialogue history. The pre-trained MASK model can be trained with semi-supervised learning, e.g., using multiple objectives from supervised tagging loss, masked tagging loss, masked language model loss, and/or a disagreement loss.