公开(公告)号：US20240046103A1

公开(公告)日：2024-02-08

申请号：US18486060

申请日：2023-10-12

Applicant: DeepMind Technologies Limited

Inventor： Jack William Rae ,  Anna Potapenko ,  Timothy Paul Lillicrap

IPC: G06N3/084 ,  G06N3/08 ,  G06F18/214 ,  G06N3/047

CPC classification number: G06N3/084 ,  G06N3/08 ,  G06F18/2148 ,  G06N3/047

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input that is a sequence to generate a network output. In one aspect, one of the methods includes, for each particular sequence of layer inputs: for each attention layer in the neural network: maintaining episodic memory data; maintaining compressed memory data; receiving a layer input to be processed by the attention layer; and applying an attention mechanism over (i) the compressed representation in the compressed memory data for the layer, (ii) the hidden states in the episodic memory data for the layer, and (iii) the respective hidden state at each of the plurality of input positions in the particular network input to generate a respective activation for each input position in the layer input.

公开(公告)号：US11829884B2

公开(公告)日：2023-11-28

申请号：US17033396

申请日：2020-09-25

Applicant: DeepMind Technologies Limited

Inventor： Jack William Rae ,  Anna Potapenko ,  Timothy Paul Lillicrap

IPC: G06N3/08 ,  G06N3/084 ,  G06F18/214 ,  G06N3/047

CPC classification number: G06N3/084 ,  G06F18/2148 ,  G06N3/047 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input that is a sequence to generate a network output. In one aspect, one of the methods includes, for each particular sequence of layer inputs: for each attention layer in the neural network: maintaining episodic memory data; maintaining compressed memory data; receiving a layer input to be processed by the attention layer; and applying an attention mechanism over (i) the compressed representation in the compressed memory data for the layer, (ii) the hidden states in the episodic memory data for the layer, and (iii) the respective hidden state at each of the plurality of input positions in the particular network input to generate a respective activation for each input position in the layer input.

公开(公告)号：US11803750B2

公开(公告)日：2023-10-31

申请号：US17019927

申请日：2020-09-14

Applicant: DeepMind Technologies Limited

Inventor： Timothy Paul Lillicrap ,  Jonathan James Hunt ,  Alexander Pritzel ,  Nicolas Manfred Otto Heess ,  Tom Erez ,  Yuval Tassa ,  David Silver ,  Daniel Pieter Wierstra

IPC: G06N3/08 ,  G06N3/006 ,  G06N3/084 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N3/006 ,  G06N3/045 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an actor neural network used to select actions to be performed by an agent interacting with an environment. One of the methods includes obtaining a minibatch of experience tuples; and updating current values of the parameters of the actor neural network, comprising: for each experience tuple in the minibatch: processing the training observation and the training action in the experience tuple using a critic neural network to determine a neural network output for the experience tuple, and determining a target neural network output for the experience tuple; updating current values of the parameters of the critic neural network using errors between the target neural network outputs and the neural network outputs; and updating the current values of the parameters of the actor neural network using the critic neural network.

公开(公告)号：US20210089829A1

公开(公告)日：2021-03-25

申请号：US17033396

申请日：2020-09-25

Applicant: DeepMind Technologies Limited

Inventor： Jack William Rae ,  Anna Potapenko ,  Timothy Paul Lillicrap

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input that is a sequence to generate a network output. In one aspect, one of the methods includes, for each particular sequence of layer inputs: for each attention layer in the neural network: maintaining episodic memory data; maintaining compressed memory data; receiving a layer input to be processed by the attention layer; and applying an attention mechanism over (i) the compressed representation in the compressed memory data for the layer, (ii) the hidden states in the episodic memory data for the layer, and (iii) the respective hidden state at each of the plurality of input positions in the particular network input to generate a respective activation for each input position in the layer input.

公开(公告)号：US10789511B2

公开(公告)日：2020-09-29

申请号：US16601324

申请日：2019-10-14

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Timothy Paul Lillicrap ,  Chia-Chun Hung ,  Joshua Simon Abramson

IPC: G06K9/62 ,  G06F11/30 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network system used to control an agent interacting with an environment to perform a specified task. One of the methods includes causing the agent to perform a task episode in which the agent attempts to perform the specified task; for each of one or more particular time steps in the sequence: generating a modified reward for the particular time step from (i) the actual reward at the time step and (ii) value predictions at one or more time steps that are more than a threshold number of time steps after the particular time step in the sequence; and training, through reinforcement learning, the neural network system using at least the modified rewards for the particular time steps.

公开(公告)号：US11875258B1

公开(公告)日：2024-01-16

申请号：US17541186

申请日：2021-12-02

Applicant: DeepMind Technologies Limited

Inventor： Nicolas Manfred Otto Heess ,  Timothy Paul Lillicrap ,  Gregory Duncan Wayne ,  Yuval Tassa

IPC: G06N3/08 ,  G06N3/006 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N3/006 ,  G06N3/044 ,  G06N3/045

Abstract: Methods, systems, and apparatus for selecting actions to be performed by an agent interacting with an environment. One system includes a high-level controller neural network, low-level controller network, and subsystem. The high-level controller neural network receives an input observation and processes the input observation to generate a high-level output defining a control signal for the low-level controller. The low-level controller neural network receives a designated component of an input observation and processes the designated component and an input control signal to generate a low-level output that defines an action to be performed by the agent in response to the input observation. The subsystem receives a current observation characterizing a current state of the environment, determines whether criteria are satisfied for generating a new control signal, and based on the determination, provides appropriate inputs to the high-level and low-level controllers for selecting an action to be performed by the agent.

公开(公告)号：US20220180147A1

公开(公告)日：2022-06-09

申请号：US17441463

申请日：2020-05-19

Applicant: DeepMind Technologies Limited

Inventor： Sergey Bartunov ,  Jack William Rae ,  Timothy Paul Lillicrap ,  Simon Osindero

IPC: G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing associative memory. In one aspect a system comprises an associative memory neural network to process an input to generate an output that defines an energy corresponding to the input. A reading subsystem retrieves stored information from the associative memory neural network. The reading subsystem performs operations including receiving a given, i.e. query, input and retrieving a data element from the associative memory neural network that is associated with the given input. The retrieving is performed by iteratively adjusting the given input using the associative memory neural network.

公开(公告)号：US20210081723A1

公开(公告)日：2021-03-18

申请号：US17035546

申请日：2020-09-28

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Timothy Paul Lillicrap ,  Chia-Chun Hung ,  Joshua Simon Abramson

IPC: G06K9/62 ,  G06F11/30 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network system used to control an agent interacting with an environment to perform a specified task. One of the methods includes causing the agent to perform a task episode in which the agent attempts to perform the specified task; for each of one or more particular time steps in the sequence: generating a modified reward for the particular time step from (i) the actual reward at the time step and (ii) value predictions at one or more time steps that are more than a threshold number of time steps after the particular time step in the sequence; and training, through reinforcement learning, the neural network system using at least the modified rewards for the particular time steps.

公开(公告)号：US20210034969A1

公开(公告)日：2021-02-04

申请号：US16766945

申请日：2019-03-11

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Chia-Chun Hung ,  David Antony Amos ,  Mehdi Mirza Mohammadi ,  Arun Ahuja ,  Timothy Paul Lillicrap

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a memory-based prediction system configured to receive an input observation characterizing a state of an environment interacted with by an agent and to process the input observation and data read from a memory to update data stored in the memory and to generate a latent representation of the state of the environment. The method comprises: for each of a plurality of time steps: processing an observation for the time step and data read from the memory to: (i) update the data stored in the memory, and (ii) generate a latent representation of the current state of the environment as of the time step; and generating a predicted return that will be received by the agent as a result of interactions with the environment after the observation for the time step is received.

公开(公告)号：US10885426B2

公开(公告)日：2021-01-05

申请号：US15396289

申请日：2016-12-30

Applicant: DeepMind Technologies Limited

Inventor： Adam Anthony Santoro ,  Daniel Pieter Wiestra ,  Timothy Paul Lillicrap ,  Sergey Bartunov ,  Ivo Danihelka

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G06F12/123

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a controller neural network that includes a Least Recently Used Access (LRUA) subsystem configured to: maintain a respective usage weight for each of a plurality of locations in the external memory, and for each of the plurality of time steps: generate a respective reading weight for each location using a read key, read data from the locations in accordance with the reading weights, generate a respective writing weight for each of the locations from a respective reading weight from a preceding time step and the respective usage weight for the location, write a write vector to the locations in accordance with the writing weights, and update the respective usage weight from the respective reading weight and the respective writing weight.