公开(公告)号：US11977967B2

公开(公告)日：2024-05-07

申请号：US17113669

申请日：2020-12-07

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Chia-Chun Hung ,  Mevlana Celaleddin Gemici ,  Adam Anthony Santoro

IPC: G06N3/06 ,  G06N3/0455 ,  G06N3/049 ,  G06N20/00 ,  G06F16/908 ,  G06N3/084

CPC classification number: G06N3/06 ,  G06N3/0455 ,  G06N3/049 ,  G06N20/00 ,  G05B2219/33025 ,  G06F16/908 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating sequences of predicted observations, for example images. In one aspect, a system comprises a controller recurrent neural network, and a decoder neural network to process a set of latent variables to generate an observation. An external memory and a memory interface subsystem is configured to, for each of a plurality of time steps, receive an updated hidden state from the controller, generate a memory context vector by reading data from the external memory using the updated hidden state, determine a set of latent variables from the memory context vector, generate a predicted observation by providing the set of latent variables to the decoder neural network, write data to the external memory using the latent variables, the updated hidden state, or both, and generate a controller input for a subsequent time step from the latent variables.

公开(公告)号：US20250051289A1

公开(公告)日：2025-02-13

申请号：US18929321

申请日：2024-10-28

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Chia-Chun Hung ,  David Antony Amos ,  Mehdi Mirza Mohammadi ,  Arun Ahuja ,  Timothy Paul Lillicrap

IPC: C07D239/47 ,  A61P35/00 ,  C07C275/36

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a memory-based prediction system configured to receive an input observation characterizing a state of an environment interacted with by an agent and to process the input observation and data read from a memory to update data stored in the memory and to generate a latent representation of the state of the environment. The method comprises: for each of a plurality of time steps: processing an observation for the time step and data read from the memory to: (i) update the data stored in the memory, and (ii) generate a latent representation of the current state of the environment as of the time step; and generating a predicted return that will be received by the agent as a result of interactions with the environment after the observation for the time step is received.

公开(公告)号：US10832134B2

公开(公告)日：2020-11-10

申请号：US15374974

申请日：2016-12-09

Applicant: DeepMind Technologies Limited

Inventor： Alexander Benjamin Graves ,  Ivo Danihelka ,  Timothy James Alexander Harley ,  Malcolm Kevin Campbell Reynolds ,  Gregory Duncan Wayne

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a memory interface subsystem that is configured to perform operations comprising determining a respective content-based weight for each of a plurality of locations in an external memory; determining a respective allocation weight for each of the plurality of locations in the external memory; determining a respective final writing weight for each of the plurality of locations in the external memory from the respective content-based weight for the location and the respective allocation weight for the location; and writing data defined by the write vector to the external memory in accordance with the final writing weights.

公开(公告)号：US20200226446A1

公开(公告)日：2020-07-16

申请号：US16831566

申请日：2020-03-26

Applicant: DeepMind Technologies Limited

Inventor： Alexander Benjamin Graves ,  Ivo Danihelka ,  Gregory Duncan Wayne

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from a first portion of a neural network output as a system output; determining one or more sets of writing weights for each of a plurality of locations in an external memory; writing data defined by a third portion of the neural network output to the external memory in accordance with the sets of writing weights; determining one or more sets of reading weights for each of the plurality of locations in the external memory from a fourth portion of the neural network output; reading data from the external memory in accordance with the sets of reading weights; and combining the data read from the external memory with a next system input to generate the next neural network input.

公开(公告)号：US12159221B2

公开(公告)日：2024-12-03

申请号：US16766945

申请日：2019-03-11

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Chia-Chun Hung ,  David Antony Amos ,  Mehdi Mirza Mohammadi ,  Arun Ahuja ,  Timothy Paul Lillicrap

IPC: G06N3/08 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a memory-based prediction system configured to receive an input observation characterizing a state of an environment interacted with by an agent and to process the input observation and data read from a memory to update data stored in the memory and to generate a latent representation of the state of the environment. The method comprises: for each of a plurality of time steps: processing an observation for the time step and data read from the memory to: (i) update the data stored in the memory, and (ii) generate a latent representation of the current state of the environment as of the time step; and generating a predicted return that will be received by the agent as a result of interactions with the environment after the observation for the time step is received.

公开(公告)号：US11080594B2

公开(公告)日：2021-08-03

申请号：US15396331

申请日：2016-12-30

Applicant: DeepMind Technologies Limited

Inventor： Ilya Sutskever ,  Ivo Danihelka ,  Alexander Benjamin Graves ,  Gregory Duncan Wayne ,  Wojciech Zaremba

IPC: G06N3/08 ,  G06F3/06 ,  G06N3/04 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory using reinforcement learning. One of the methods includes providing an output derived from the system output portion of the neural network output as a system output in the sequence of system outputs; selecting a memory access process from a predetermined set of memory access processes for accessing the external memory from the reinforcement learning portion of the neural network output; writing and reading data from locations in the external memory in accordance with the selected memory access process using the differentiable portion of the neural network output; and combining the data read from the external memory with a next system input in the sequence of system inputs to generate a next neural network input in the sequence of neural network inputs.

公开(公告)号：US20210117801A1

公开(公告)日：2021-04-22

申请号：US17093373

申请日：2020-11-09

Applicant: DeepMind Technologies Limited

Inventor： Alexander Benjamin Graves ,  Ivo Danihelka ,  Timothy James Alexander Harley ,  Malcolm Kevin Campbell Reynolds ,  Gregory Duncan Wayne

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a memory interface subsystem that is configured to perform operations comprising determining a respective content-based weight for each of a plurality of locations in an external memory; determining a respective allocation weight for each of the plurality of locations in the external memory; determining a respective final writing weight for each of the plurality of locations in the external memory from the respective content-based weight for the location and the respective allocation weight for the location; and writing data defined by the write vector to the external memory in accordance with the final writing weights.

公开(公告)号：US10789511B2

公开(公告)日：2020-09-29

申请号：US16601324

申请日：2019-10-14

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Timothy Paul Lillicrap ,  Chia-Chun Hung ,  Joshua Simon Abramson

IPC: G06K9/62 ,  G06F11/30 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network system used to control an agent interacting with an environment to perform a specified task. One of the methods includes causing the agent to perform a task episode in which the agent attempts to perform the specified task; for each of one or more particular time steps in the sequence: generating a modified reward for the particular time step from (i) the actual reward at the time step and (ii) value predictions at one or more time steps that are more than a threshold number of time steps after the particular time step in the sequence; and training, through reinforcement learning, the neural network system using at least the modified rewards for the particular time steps.

公开(公告)号：US10691997B2

公开(公告)日：2020-06-23

申请号：US14977201

申请日：2015-12-21

Applicant: DeepMind Technologies Limited

Inventor： Alexander Benjamin Graves ,  Ivo Danihelka ,  Gregory Duncan Wayne

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks to generate additional outputs. One of the systems includes a neural network and a sequence processing subsystem, wherein the sequence processing subsystem is configured to perform operations comprising, for each of the system inputs in a sequence of system inputs: receiving the system input; generating an initial neural network input from the system input; causing the neural network to process the initial neural network input to generate an initial neural network output for the system input; and determining, from a first portion of the initial neural network output for the system input, whether or not to cause the neural network to generate one or more additional neural network outputs for the system input.

公开(公告)号：US11875258B1

公开(公告)日：2024-01-16

申请号：US17541186

申请日：2021-12-02

Applicant: DeepMind Technologies Limited

Inventor： Nicolas Manfred Otto Heess ,  Timothy Paul Lillicrap ,  Gregory Duncan Wayne ,  Yuval Tassa

IPC: G06N3/08 ,  G06N3/006 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N3/006 ,  G06N3/044 ,  G06N3/045

Abstract: Methods, systems, and apparatus for selecting actions to be performed by an agent interacting with an environment. One system includes a high-level controller neural network, low-level controller network, and subsystem. The high-level controller neural network receives an input observation and processes the input observation to generate a high-level output defining a control signal for the low-level controller. The low-level controller neural network receives a designated component of an input observation and processes the designated component and an input control signal to generate a low-level output that defines an action to be performed by the agent in response to the input observation. The subsystem receives a current observation characterizing a current state of the environment, determines whether criteria are satisfied for generating a new control signal, and based on the determination, provides appropriate inputs to the high-level and low-level controllers for selecting an action to be performed by the agent.