公开(公告)号：US11010663B2

公开(公告)日：2021-05-18

申请号：US15395553

申请日：2016-12-30

Applicant: DeepMind Technologies Limited

Inventor： Ivo Danihelka ,  Nal Emmerich Kalchbrenner ,  Gregory Duncan Wayne ,  Benigno Uría-Martínez ,  Alexander Benjamin Graves

IPC: G06N3/08 ,  G06N3/04

Abstract: Systems, methods, and apparatus, including computer programs encoded on a computer storage medium, related to associative long short-term memory (LSTM) neural network layers configured to maintain N copies of an internal state for the associative LSTM layer, N being an integer greater than one. In one aspect, a system includes a recurrent neural network including an associative LSTM layer, wherein the associative LSTM layer is configured to, for each time step, receive a layer input, update each of the N copies of the internal state using the layer input for the time step and a layer output generated by the associative LSTM layer for a preceding time step, and generate a layer output for the time step using the N updated copies of the internal state.

公开(公告)号：US10650302B2

公开(公告)日：2020-05-12

申请号：US14885086

申请日：2015-10-16

Applicant: DeepMind Technologies Limited

Inventor： Alexander Benjamin Graves ,  Ivo Danihelka ,  Gregory Duncan Wayne

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from a first portion of a neural network output as a system output; determining one or more sets of writing weights for each of a plurality of locations in an external memory; writing data defined by a third portion of the neural network output to the external memory in accordance with the sets of writing weights; determining one or more sets of reading weights for each of the plurality of locations in the external memory from a fourth portion of the neural network output; reading data from the external memory in accordance with the sets of reading weights; and combining the data read from the external memory with a next system input to generate the next neural network input.

公开(公告)号：US20200117956A1

公开(公告)日：2020-04-16

申请号：US16601324

申请日：2019-10-14

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Timothy Paul Lillicrap ,  Chia-Chun Hung ,  Joshua Simon Abramson

IPC: G06K9/62 ,  G06N3/08 ,  G06F11/30

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network system used to control an agent interacting with an environment to perform a specified task. One of the methods includes causing the agent to perform a task episode in which the agent attempts to perform the specified task; for each of one or more particular time steps in the sequence: generating a modified reward for the particular time step from (i) the actual reward at the time step and (ii) value predictions at one or more time steps that are more than a threshold number of time steps after the particular time step in the sequence; and training, through reinforcement learning, the neural network system using at least the modified rewards for the particular time steps.

公开(公告)号：US20200090042A1

公开(公告)日：2020-03-19

申请号：US16688934

申请日：2019-11-19

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Joshua Merel ,  Ziyu Wang ,  Nicolas Manfred Otto Heess ,  Joao Ferdinando Gomes de Freitas ,  Scott Ellison Reed

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network used to select actions to be performed by an agent interacting with an environment. One of the methods includes: obtaining data identifying a set of trajectories, each trajectory comprising a set of observations characterizing a set of states of the environment and corresponding actions performed by another agent in response to the states; obtaining data identifying an encoder that maps the observations onto embeddings for use in determining a set of imitation trajectories; determining, for each trajectory, a corresponding embedding by applying the encoder to the trajectory; determining a set of imitation trajectories by applying a policy defined by the neural network to the embedding for each trajectory; and adjusting parameters of the neural network based on the set of trajectories, the set of imitation trajectories and the embeddings.

公开(公告)号：US20190324988A1

公开(公告)日：2019-10-24

申请号：US16459113

申请日：2019-07-01

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Chia-Chun Hung ,  Mevlana Celaleddin Gemici ,  Adam Anthony Santoro

IPC: G06F16/908 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating sequences of predicted observations, for example images. In one aspect, a system comprises a controller recurrent neural network, and a decoder neural network to process a set of latent variables to generate an observation. An external memory and a memory interface subsystem is configured to, for each of a plurality of time steps, receive an updated hidden state from the controller, generate a memory context vector by reading data from the external memory using the updated hidden state, determine a set of latent variables from the memory context vector, generate a predicted observation by providing the set of latent variables to the decoder neural network, write data to the external memory using the latent variables, the updated hidden state, or both, and generate a controller input for a subsequent time step from the latent variables.