公开(公告)号：US20160098632A1

公开(公告)日：2016-04-07

申请号：US14877071

申请日：2015-10-07

Applicant: Google Inc.

Inventor： Ilya Sutskever ,  Wojciech Zaremba

IPC: G06N3/08

CPC classification number: G06N3/08 ,  G06N3/0445 ,  G06N3/10

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes obtaining partitioned training data for the neural network, wherein the partitioned training data comprises a plurality of training items each of which is assigned to a respective one of a plurality of partitions, wherein each partition is associated with a respective difficulty level; and training the neural network on each of the partitions in a sequence from a partition associated with an easiest difficulty level to a partition associated with a hardest difficulty level, wherein, for each of the partitions, training the neural network comprises: training the neural network on a sequence of training items that includes training items selected from the training items in the partition interspersed with training items selected from the training items in all of the partitions.

Abstract translation: 方法，系统和装置，包括在计算机存储介质上编码的用于训练神经网络的计算机程序。 其中一种方法包括获得用于神经网络的分区训练数据，其中分区训练数据包括多个训练项目，每个训练项目分配给多个分区中的相应一个分区，其中每个分区与相应的难度等级相关联 ; 并且将每个分区上的神经网络从与最简单难度级别相关联的分区中的序列训练到与最困难级别相关联的分区，其中，对于每个分区，训练神经网络包括：训练神经网络 在训练项目的序列上，其包括从分配有从所有分区中的训练项目中选择的训练项目的分区中的训练项目中选择的训练项目。

公开(公告)号：US20160117316A1

公开(公告)日：2016-04-28

申请号：US14921925

申请日：2015-10-23

Applicant: Google Inc.

Inventor： Quoc V. Le ,  Minh-Thang Luong ,  Ilya Sutskever ,  Oriol Vinyals ,  Wojciech Zaremba

IPC: G06F17/28 ,  G10L15/02 ,  G10L15/16 ,  G06F7/02 ,  G06F7/10

CPC classification number: G06F17/2881 ,  G06F7/023 ,  G06F7/10 ,  G06F17/2735 ,  G06F17/2818 ,  G06F17/2827 ,  G06N3/0445 ,  G06N3/0454 ,  G10L15/02 ,  G10L15/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural translation systems with rare word processing. One of the methods is a method training a neural network translation system to track the source in source sentences of unknown words in target sentences, in a source language and a target language, respectively and includes deriving alignment data from a parallel corpus, the alignment data identifying, in each pair of source and target language sentences in the parallel corpus, aligned source and target words; annotating the sentences in the parallel corpus according to the alignment data and a rare word model to generate a training dataset of paired source and target language sentences; and training a neural network translation model on the training dataset.

Abstract translation: 方法，系统和装置，包括在计算机存储介质上编码的计算机程序，用于具有罕见文字处理的神经翻译系统。 其中一种方法是训练神经网络翻译系统，以分别在源语言和目标语言中跟踪目标语句中的未知单词的源语句中的源，并且包括从并行语料库导出对齐数据，对齐数据 在平行语料库中的每对源和目标语言句子中识别对齐的源词和目标词; 根据对齐数据和平行语料库中的句子注释罕见词模型，以生成配对的源语言和目标语言句子的训练数据集; 并在训练数据集上训练神经网络翻译模型。

公开(公告)号：US20170323201A1

公开(公告)日：2017-11-09

申请号：US15396331

申请日：2016-12-30

Applicant: Google Inc.

Inventor： Ilya Sutskever ,  Ivo Danihelka ,  Alexander Benjamin Graves ,  Gregory Duncan Wayne ,  Wojciech Zaremba

IPC: G06N3/08 ,  G06F3/06

CPC classification number: G06N3/08 ,  G06F3/0604 ,  G06F3/0653 ,  G06F3/0673 ,  G06N3/0445 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory using reinforcement learning. One of the methods includes providing an output derived from the system output portion of the neural network output as a system output in the sequence of system outputs; selecting a memory access process from a predetermined set of memory access processes for accessing the external memory from the reinforcement learning portion of the neural network output; writing and reading data from locations in the external memory in accordance with the selected memory access process using the differentiable portion of the neural network output; and combining the data read from the external memory with a next system input in the sequence of system inputs to generate a next neural network input in the sequence of neural network inputs.