公开(公告)号：US11354565B2

公开(公告)日：2022-06-07

申请号：US15853530

申请日：2017-12-22

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06N20/00 ,  G06F40/30 ,  G06K9/62 ,  G06F15/76 ,  G06F40/169 ,  G06N3/08 ,  G06N5/04

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US20180268287A1

公开(公告)日：2018-09-20

申请号：US15853530

申请日：2017-12-22

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06K9/62

CPC classification number: G06N3/0454 ,  G06F15/76 ,  G06F17/241 ,  G06F17/2785 ,  G06K9/6262 ,  G06K9/6267 ,  G06K9/6268 ,  G06K9/6271 ,  G06N3/0445 ,  G06N3/0472 ,  G06N3/0481 ,  G06N3/084 ,  G06N5/04 ,  G06N20/00

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US11080595B2

公开(公告)日：2021-08-03

申请号：US15420801

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： James Bradbury ,  Stephen Joseph Merity ,  Caiming Xiong ,  Richard Socher

IPC: G06N3/04 ,  G06N3/08 ,  G06F40/30 ,  G06F40/44 ,  G06F40/216 ,  G06F17/16 ,  G06N3/10 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30 ,  G06F40/00

Abstract: The technology disclosed provides a quasi-recurrent neural network (QRNN) encoder-decoder model that alternates convolutional layers, which apply in parallel across timesteps, and minimalist recurrent pooling layers that apply in parallel across feature dimensions.

公开(公告)号：US20180268298A1

公开(公告)日：2018-09-20

申请号：US15853570

申请日：2017-12-22

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62 ,  G06F17/24 ,  G06F17/27 ,  G06N5/04 ,  G06F15/18

CPC classification number: G06N3/0454 ,  G06F15/76 ,  G06F17/241 ,  G06F17/2785 ,  G06K9/6262 ,  G06K9/6267 ,  G06K9/6268 ,  G06K9/6271 ,  G06N3/0445 ,  G06N3/0472 ,  G06N3/0481 ,  G06N3/084 ,  G06N5/04 ,  G06N20/00

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US11822897B2

公开(公告)日：2023-11-21

申请号：US17463227

申请日：2021-08-31

Applicant: salesforce.com, inc.

Inventor： Kazuma Hashimoto ,  Raffaella Buschiazzo ,  James Bradbury ,  Teresa Anna Marshall ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/58 ,  G06N3/08

CPC classification number: G06F40/58 ,  G06N3/08

Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.

公开(公告)号：US11537801B2

公开(公告)日：2022-12-27

申请号：US17214691

申请日：2021-03-26

Applicant: salesforce.com, inc.

Inventor： Kazuma Hashimoto ,  Raffaella Buschiazzo ,  James Bradbury ,  Teresa Marshall ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/58 ,  G06N3/08

Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.

公开(公告)号：US20220164635A1

公开(公告)日：2022-05-26

申请号：US17670368

申请日：2022-02-11

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06K9/62 ,  G06N20/00 ,  G06F15/76 ,  G06F40/30 ,  G06F40/169 ,  G06N3/08 ,  G06N5/04

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US20210397799A1

公开(公告)日：2021-12-23

申请号：US17463227

申请日：2021-08-31

Applicant: salesforce.com, inc.

Inventor： Kazuma Hashimoto ,  Raffaella Buschiazzo ,  James Bradbury ,  Teresa Anna Marshall ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/58

Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.

公开(公告)号：US11580359B2

公开(公告)日：2023-02-14

申请号：US16664508

申请日：2019-10-25

Applicant: salesforce.com, inc.

Inventor： Stephen Joseph Merity ,  Caiming Xiong ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06N3/084 ,  G06F40/284 ,  G06N3/08 ,  G06N7/00

Abstract: The technology disclosed provides a so-called “pointer sentinel mixture architecture” for neural network sequence models that has the ability to either reproduce a token from a recent context or produce a token from a predefined vocabulary. In one implementation, a pointer sentinel-LSTM architecture achieves state of the art language modeling performance of 70.9 perplexity on the Penn Treebank dataset, while using far fewer parameters than a standard softmax LSTM.

公开(公告)号：US10963652B2

公开(公告)日：2021-03-30

申请号：US16264392

申请日：2019-01-31

Applicant: salesforce.com, inc.

Inventor： Kazuma Hashimoto ,  Raffaella Buschiazzo ,  James Bradbury ,  Teresa Marshall ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/58 ,  G06N3/08

Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.