公开(公告)号：US12277487B2

公开(公告)日：2025-04-15

申请号：US17441463

申请日：2020-05-19

Applicant: DeepMind Technologies Limited

Inventor： Sergey Bartunov ,  Jack William Rae ,  Timothy Paul Lillicrap ,  Simon Osindero

IPC: G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing associative memory. In one aspect a system comprises an associative memory neural network to process an input to generate an output that defines an energy corresponding to the input. A reading subsystem retrieves stored information from the associative memory neural network. The reading subsystem performs operations including receiving a given, i.e. query, input and retrieving a data element from the associative memory neural network that is associated with the given input. The retrieving is performed by iteratively adjusting the given input using the associative memory neural network.

公开(公告)号：US20250093970A1

公开(公告)日：2025-03-20

申请号：US18967935

申请日：2024-12-04

Applicant: DeepMind Technologies Limited

Inventor： Peter Conway Humphreys ,  Timothy Paul Lillicrap ,  Tobias Markus Pohlen ,  Adam Anthony Santoro

IPC: G06F3/033 ,  G06F3/023 ,  G06F40/284

Abstract: A computer-implemented method for controlling a particular computer to execute a task is described. The method includes receiving a control input comprising a visual input, the visual input including one or more screen frames of a computer display that represent at least a current state of the particular computer; processing the control input using a neural network to generate one or more control outputs that are used to control the particular computer to execute the task, in which the one or more control outputs include an action type output that specifies at least one of a pointing device action or a keyboard action to be performed to control the particular computer; determining one or more actions from the one or more control outputs; and executing the one or more actions to control the particular computer.

公开(公告)号：US20240320506A1

公开(公告)日：2024-09-26

申请号：US18698890

申请日：2022-10-05

Applicant: DeepMind Technologies Limited

Inventor： Anirudh Goyal ,  Andrea Banino ,  Abram Luke Friesen ,  Theophane Guillaume Weber ,  Adrià Puigdomènech Badia ,  Nan Ke ,  Simon Osindero ,  Timothy Paul Lillicrap ,  Charles Blundell

IPC: G06N3/092 ,  G06N3/044 ,  G06N3/0455 ,  G06N3/084

CPC classification number: G06N3/092 ,  G06N3/044 ,  G06N3/0455 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling a reinforcement learning agent in an environment to perform a task using a retrieval-augmented action selection process. One of the methods includes receiving a current observation characterizing a current state of the environment; processing an encoder network input comprising the current observation to determine a policy neural network hidden state that corresponds to the current observation; maintaining a plurality of trajectories generated as a result of the reinforcement learning agent interacting with the environment; selecting one or more trajectories from the plurality of trajectories; updating the policy neural network hidden state using update data determined from the one or more selected trajectories; and processing the updated hidden state using a policy neural network to generate a policy output that specifies an action to be performed by the agent in response to the current observation.

公开(公告)号：US12032523B2

公开(公告)日：2024-07-09

申请号：US16818895

申请日：2020-03-13

Applicant: DeepMind Technologies Limited

Inventor： Yan Wu ,  Timothy Paul Lillicrap ,  Mihaela Rosca

IPC: G06F16/174 ,  G06N3/045 ,  G06N3/08

CPC classification number: G06F16/1744 ,  G06N3/045 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compressed sensing using neural networks. One of the methods includes receiving an input measurement of an input data item; for each of one or more optimization steps: processing a latent representation using a generator neural network to generate a candidate reconstructed data item, processing the candidate reconstructed data item using a measurement neural network to generate a measurement of the candidate reconstructed data item, and updating the latent representation to reduce an error between the measurement and the input measurement; and processing the latent representation after the one or more optimization steps using the generator neural network to generate a reconstruction of the input data item.

公开(公告)号：US20250051289A1

公开(公告)日：2025-02-13

申请号：US18929321

申请日：2024-10-28

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Chia-Chun Hung ,  David Antony Amos ,  Mehdi Mirza Mohammadi ,  Arun Ahuja ,  Timothy Paul Lillicrap

IPC: C07D239/47 ,  A61P35/00 ,  C07C275/36

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a memory-based prediction system configured to receive an input observation characterizing a state of an environment interacted with by an agent and to process the input observation and data read from a memory to update data stored in the memory and to generate a latent representation of the state of the environment. The method comprises: for each of a plurality of time steps: processing an observation for the time step and data read from the memory to: (i) update the data stored in the memory, and (ii) generate a latent representation of the current state of the environment as of the time step; and generating a predicted return that will be received by the agent as a result of interactions with the environment after the observation for the time step is received.

公开(公告)号：US11983617B2

公开(公告)日：2024-05-14

申请号：US17102318

申请日：2020-11-23

Applicant: DeepMind Technologies Limited

Inventor： Jack William Rae ,  Timothy Paul Lillicrap ,  Sergey Bartunov

IPC: G06N3/045 ,  G06F16/22 ,  G06N3/08

CPC classification number: G06N3/045 ,  G06F16/2272 ,  G06N3/08

Abstract: A system for compressed data storage using a neural network. The system comprises a memory comprising a plurality of memory locations configured to store data; a query neural network configured to process a representation of an input data item to generate a query; an immutable key data store comprising key data for indexing the plurality of memory locations; an addressing system configured to process the key data and the query to generate a weighting associated with the plurality of memory locations; a memory read system configured to generate output memory data from the memory based upon the generated weighting associated with the plurality of memory locations and the data stored at the plurality of memory locations; and a memory write system configured to write received write data to the memory based upon the generated weighting associated with the plurality of memory locations.

公开(公告)号：US11741334B2

公开(公告)日：2023-08-29

申请号：US16882373

申请日：2020-05-22

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06N3/08 ,  G06N3/006 ,  G06N3/088 ,  G06F18/21 ,  G06F18/214 ,  G06N3/045

CPC classification number: G06N3/006 ,  G06F18/2148 ,  G06F18/2185 ,  G06N3/045 ,  G06N3/08 ,  G06N3/088

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.

公开(公告)号：US20200293497A1

公开(公告)日：2020-09-17

申请号：US16818895

申请日：2020-03-13

Applicant: DeepMind Technologies Limited

Inventor： Yan Wu ,  Timothy Paul Lillicrap ,  Mihaela Rosca

IPC: G06F16/174 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compressed sensing using neural networks. One of the methods includes receiving an input measurement of an input data item; for each of one or more optimization steps: processing a latent representation using a generator neural network to generate a candidate reconstructed data item, processing the candidate reconstructed data item using a measurement neural network to generate a measurement of the candidate reconstructed data item, and updating the latent representation to reduce an error between the measurement and the input measurement; and processing the latent representation after the one or more optimization steps using the generator neural network to generate a reconstruction of the input data item.

公开(公告)号：US12159221B2

公开(公告)日：2024-12-03

申请号：US16766945

申请日：2019-03-11

Applicant: DeepMind Technologies Limited

Inventor： Gregory Duncan Wayne ,  Chia-Chun Hung ,  David Antony Amos ,  Mehdi Mirza Mohammadi ,  Arun Ahuja ,  Timothy Paul Lillicrap

IPC: G06N3/08 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a memory-based prediction system configured to receive an input observation characterizing a state of an environment interacted with by an agent and to process the input observation and data read from a memory to update data stored in the memory and to generate a latent representation of the state of the environment. The method comprises: for each of a plurality of time steps: processing an observation for the time step and data read from the memory to: (i) update the data stored in the memory, and (ii) generate a latent representation of the current state of the environment as of the time step; and generating a predicted return that will be received by the agent as a result of interactions with the environment after the observation for the time step is received.

公开(公告)号：US20240062035A1

公开(公告)日：2024-02-22

申请号：US18351440

申请日：2023-07-12

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06N3/006 ,  G06N3/08 ,  G06N3/088 ,  G06F18/21 ,  G06F18/214 ,  G06N3/045

CPC classification number: G06N3/006 ,  G06N3/08 ,  G06N3/088 ,  G06F18/2185 ,  G06F18/2148 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.