公开(公告)号：US11625604B2

公开(公告)日：2023-04-11

申请号：US16641751

申请日：2018-10-29

Applicant: DeepMind Technologies Limited

Inventor： David Budden ,  Gabriel Barth-Maron ,  John Quan ,  Daniel George Horgan

IPC: G06N3/08 ,  G06N3/04 ,  G06N20/00 ,  G06N3/088

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. One of the systems includes (i) a plurality of actor computing units, in which each of the actor computing units is configured to maintain a respective replica of the action selection neural network and to perform a plurality of actor operations, and (ii) one or more learner computing units, in which each of the one or more learner computing units is configured to perform a plurality of learner operations.

公开(公告)号：US20200265305A1

公开(公告)日：2020-08-20

申请号：US16641751

申请日：2018-10-29

Applicant: DeepMind Technologies Limited

Inventor： David Budden ,  Gabriel Barth-Maron ,  John Quan ,  Daniel George Horgan

IPC: G06N3/08 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. One of the systems includes (i) a plurality of actor computing units, in which each of the actor computing units is configured to maintain a respective replica of the action selection neural network and to perform a plurality of actor operations, and (ii) one or more learner computing units, in which each of the one or more learner computing units is configured to perform a plurality of learner operations.

公开(公告)号：US12277497B2

公开(公告)日：2025-04-15

申请号：US18131753

申请日：2023-04-06

Applicant: DeepMind Technologies Limited

Inventor： David Budden ,  Gabriel Barth-Maron ,  John Quan ,  Daniel George Horgan

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/088 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. One of the systems includes (i) a plurality of actor computing units, in which each of the actor computing units is configured to maintain a respective replica of the action selection neural network and to perform a plurality of actor operations, and (ii) one or more learner computing units, in which each of the one or more learner computing units is configured to perform a plurality of learner operations.

公开(公告)号：US11663475B2

公开(公告)日：2023-05-30

申请号：US17945622

申请日：2022-09-15

Applicant: DeepMind Technologies Limited

Inventor： David Budden ,  Matthew William Hoffman ,  Gabriel Barth-Maron

IPC: G06N3/08 ,  G06N3/045 ,  G06N3/04

CPC classification number: G06N3/08 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network that is used to select actions to be performed by a reinforcement learning agent interacting with an environment. In particular, the actions are selected from a continuous action space and the system trains the action selection neural network jointly with a distribution Q network that is used to update the parameters of the action selection neural network.

公开(公告)号：US20230061411A1

公开(公告)日：2023-03-02

申请号：US17410689

申请日：2021-08-24

Applicant: DeepMind Technologies Limited

Inventor： Tom Erez ,  Alexander Novikov ,  Emilio Parisotto ,  Jack William Rae ,  Konrad Zolna ,  Misha Man Ray Denil ,  Joao Ferdinando Gomes de Freitas ,  Oriol Vinyals ,  Scott Ellison Reed ,  Sergio Gomez ,  Ashley Deloris Edwards ,  Jacob Bruce ,  Gabriel Barth-Maron

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent to interact with an environment using an action selection neural network. In one aspect, a method comprises, at each time step in a sequence of time steps: generating a current representation of a state of a task being performed by the agent in the environment as of the current time step as a sequence of data elements; autoregressively generating a sequence of data elements representing a current action to be performed by the agent at the current time step; and after autoregressively generating the sequence of data elements representing the current action, causing the agent to perform the current action at the current time step.

公开(公告)号：US20200293883A1

公开(公告)日：2020-09-17

申请号：US16759519

申请日：2018-10-29

Applicant: DeepMind Technologies Limited

Inventor： David Budden ,  Matthew William Hoffman ,  Gabriel Barth-Maron

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network that is used to select actions to be performed by a reinforcement learning agent interacting with an environment. In particular, the actions are selected from a continuous action space and the system trains the action selection neural network jointly with a distribution Q network that is used to update the parameters of the action selection neural network.

公开(公告)号：US20190354813A1

公开(公告)日：2019-11-21

申请号：US16528260

申请日：2019-07-31

Applicant: DeepMind Technologies Limited

Inventor： Martin Riedmiller ,  Roland Hafner ,  Mel Vecerik ,  Timothy Paul Lillicrap ,  Thomas Lampe ,  Ivaylo Popov ,  Gabriel Barth-Maron ,  Nicolas Manfred Otto Heess

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.