公开(公告)号：US20180144208A1

公开(公告)日：2018-05-24

申请号：US15817161

申请日：2017-11-17

Applicant: salesforce.com, inc.

Inventor： Jiasen LU ,  Caiming XIONG ,  Richard SOCHER

IPC: G06K9/46 ,  G06K9/00 ,  G06F17/24 ,  G06F17/27 ,  G06K9/48 ,  G06K9/66 ,  G06N3/08

Abstract: The technology disclosed presents a novel spatial attention model that uses current hidden state information of a decoder long short-term memory (LSTM) to guide attention and to extract spatial image features for use in image captioning. The technology disclosed also presents a novel adaptive attention model for image captioning that mixes visual information from a convolutional neural network (CNN) and linguistic information from an LSTM. At each timestep, the adaptive attention model automatically decides how heavily to rely on the image, as opposed to the linguistic model, to emit the next caption word. The technology disclosed further adds a new auxiliary sentinel gate to an LSTM architecture and produces a sentinel LSTM (Sn-LSTM). The sentinel gate produces a visual sentinel at each timestep, which is an additional representation, derived from the LSTM's memory, of long and short term visual and linguistic information.

公开(公告)号：US20180129931A1

公开(公告)日：2018-05-10

申请号：US15420801

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： James BRADBURY ,  Stephen Joseph MERITY ,  Caiming XIONG ,  Richard SOCHER

IPC: G06N3/04

CPC classification number: G06N3/08 ,  G06F17/16 ,  G06F17/20 ,  G06F17/2715 ,  G06F17/2785 ,  G06F17/2818 ,  G06N3/04 ,  G06N3/0445 ,  G06N3/0454 ,  G06N3/10 ,  G10L15/16 ,  G10L15/18 ,  G10L15/1815 ,  G10L25/30

Abstract: The technology disclosed provides a quasi-recurrent neural network (QRNN) encoder-decoder model that alternates convolutional layers, which apply in parallel across timesteps, and minimalist recurrent pooling layers that apply in parallel across feature dimensions.

公开(公告)号：US20180121799A1

公开(公告)日：2018-05-03

申请号：US15421431

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： Kazuma HASHIMOTO ,  Caiming XIONG ,  Richard SOCHER

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063

CPC classification number: G06N3/04 ,  G06F17/20 ,  G06F17/2705 ,  G06F17/2715 ,  G06F17/274 ,  G06F17/277 ,  G06F17/2785 ,  G06N3/0445 ,  G06N3/0454 ,  G06N3/0472 ,  G06N3/063 ,  G06N3/08 ,  G06N3/084 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30

Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.