公开(公告)号：US20240062035A1

公开(公告)日：2024-02-22

申请号：US18351440

申请日：2023-07-12

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06N3/006 ,  G06N3/08 ,  G06N3/088 ,  G06F18/21 ,  G06F18/214 ,  G06N3/045

CPC classification number: G06N3/006 ,  G06N3/08 ,  G06N3/088 ,  G06F18/2185 ,  G06F18/2148 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.

公开(公告)号：US11741334B2

公开(公告)日：2023-08-29

申请号：US16882373

申请日：2020-05-22

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06N3/08 ,  G06N3/006 ,  G06N3/088 ,  G06F18/21 ,  G06F18/214 ,  G06N3/045

CPC classification number: G06N3/006 ,  G06F18/2148 ,  G06F18/2185 ,  G06N3/045 ,  G06N3/08 ,  G06N3/088

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.

公开(公告)号：US20220237488A1

公开(公告)日：2022-07-28

申请号：US17613687

申请日：2020-05-22

Applicant: DeepMind Technologies Limited

Inventor： Markus Wulfmeier ,  Abbas Abdolmaleki ,  Roland Hafner ,  Jost Tobias Springenberg ,  Nicolas Manfred Otto Heess ,  Martin Riedmiller

IPC: G06N7/00 ,  G06N3/04 ,  G06N20/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes obtaining an observation characterizing a current state of the environment and data identifying a task currently being performed by the agent; processing the observation and the data identifying the task using a high-level controller to generate a high-level probability distribution that assigns a respective probability to each of a plurality of low-level controllers; processing the observation using each of the plurality of low-level controllers to generate, for each of the plurality of low-level controllers, a respective low-level probability distribution; generating a combined probability distribution; and selecting, using the combined probability distribution, an action from the space of possible actions to be performed by the agent in response to the observation.

公开(公告)号：US20190354813A1

公开(公告)日：2019-11-21

申请号：US16528260

申请日：2019-07-31

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.

公开(公告)号：US20240312657A1

公开(公告)日：2024-09-19

申请号：US18572914

申请日：2022-07-08

Applicant: DeepMind Technologies Limited

Inventor： Jonas Degrave ,  Federico Alberto Alfredo Felici ,  Jonas Buchli ,  Michael Peter Neunert ,  Brendan Daniel Tracey ,  Francesco Carpanese ,  Timo Victor Ewalds ,  Roland Hafner ,  Martin Riedmiller

IPC: G21D3/00 ,  G21B1/05

CPC classification number: G21D3/001 ,  G21B1/05

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating control signals for controlling a magnetic field for confining plasma in a chamber of a magnetic confinement device. One of the methods includes, for each of a plurality of time steps, obtaining an observation characterizing a current state of the plasma in the chamber of the magnetic confinement device, processing an input including the observation using a plasma confinement neural network to generate a magnetic control output that characterizes control signals for controlling the magnetic field of the magnetic confinement device, and generating the control signals for controlling the magnetic field of the magnetic confinement device based on the magnetic control output.

公开(公告)号：US11893480B1

公开(公告)日：2024-02-06

申请号：US16289531

申请日：2019-02-28

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner

IPC: G06N3/08 ,  G06N3/04 ,  G06N7/01

CPC classification number: G06N3/08 ,  G06N3/04 ,  G06N7/01

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning with scheduled auxiliary tasks. In one aspect, a method includes maintaining data specifying parameter values for a primary policy neural network and one or more auxiliary neural networks; at each of a plurality of selection time steps during a training episode comprising a plurality of time steps: receiving an observation, selecting a current task for the selection time step using a task scheduling policy, processing an input comprising the observation using the policy neural network corresponding to the selected current task to select an action to be performed by the agent in response to the observation, and causing the agent to perform the selected action.

公开(公告)号：US20200285909A1

公开(公告)日：2020-09-10

申请号：US16882373

申请日：2020-05-22

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06K9/62 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.

公开(公告)号：US10664725B2

公开(公告)日：2020-05-26

申请号：US16528260

申请日：2019-07-31

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06N3/08 ,  G06K9/62 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.