-
公开(公告)号:US20250124297A1
公开(公告)日:2025-04-17
申请号:US18834208
申请日:2023-01-30
Applicant: DeepMind Technologies Limited
Inventor: Mark Daniel Rowland , Shantanu Yogeshraj Thakoor , Andre da Motta Salles Barreto , Diana Luiza Borsa , William Clinton Dabney , Remi Munos
IPC: G06N3/092 , G06N3/0455
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling a reinforcement learning agent in an environment. One of the methods may include maintaining data specifying a base policy set comprising a plurality of base policies for controlling the agent; receiving a current observation characterizing a current state of the environment; generating, for each of the plurality of base policies, one or more predicted future observations characterizing respective future states of the environment that are subsequent to the current state of the environment; using the predicted future observations generated for the plurality of base policies to determine a respective estimated value for each composite policy in a composite policy set with respect to the current state of the environment; and selecting an action using the respective estimated values for the composite policies.
-
公开(公告)号:US20200327405A1
公开(公告)日:2020-10-15
申请号:US16303501
申请日:2017-05-18
Applicant: DEEPMIND TECHNOLOGIES LIMITED
Inventor: Marc Gendron-Bellemare , Remi Munos , Srinivasan Sriram
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network used to select actions to be performed by an agent interacting with an environment. One of the methods includes obtaining data identifying (i) a first observation characterizing a first state of the environment, (ii) an action performed by the agent in response to the first observation, and (iii) an actual reward received resulting from the agent performing the action in response to the first observation; determining a pseudo-count for the first observation; determining an exploration reward bonus that incentivizes the agent to explore the environment from the pseudo-count for the first observation; generating a combined reward from the actual reward and the exploration reward bonus; and adjusting current values of the parameters of the neural network using the combined reward.
-
公开(公告)号:US20250068919A1
公开(公告)日:2025-02-27
申请号:US18238400
申请日:2023-08-25
Applicant: DeepMind Technologies Limited
Inventor: Daniel Jarrett , Corentin Tallec , Florent Altché , Thomas Mesnard , Remi Munos , Michal Valko
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network used to select actions to be performed by an agent interacting with an environment. Implementations of the method model unpredictable aspects of the future, using hindsight. They use this information to disentangle inherently unpredictable, aleatoric variation, from epistemic uncertainty that arises from lack of knowledge of the environment. They then use the epistemic uncertainty, which relates to in principle predictable aspects of the environment, as a source of intrinsic reward to drive curiosity, i.e. exploration of the environment by the agent.
-
公开(公告)号:US20240256883A1
公开(公告)日:2024-08-01
申请号:US18424561
申请日:2024-01-26
Applicant: DeepMind Technologies Limited
Inventor: Thomas Mesnard , Remi Munos , Alaa Saade , Yunhao Tang , Mark Daniel Rowland , Theophane Guillaume Weber , Wenqi Chen
IPC: G06N3/092
CPC classification number: G06N3/092
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network used to select actions to be performed by an agent interacting with an environment. Implementations of the system can take into account a level of luck in the environment, and hence whilst learning can account for outcomes that were caused by external factors as well as those dependent on the actions of the agent.
-
5.
公开(公告)号:US11868894B2
公开(公告)日:2024-01-09
申请号:US18149771
申请日:2023-01-04
Applicant: DeepMind Technologies Limited
Inventor: Hubert Josef Soyer , Lasse Espeholt , Karen Simonyan , Yotam Doron , Vlad Firoiu , Volodymyr Mnih , Koray Kavukcuoglu , Remi Munos , Thomas Ward , Timothy James Alexander Harley , Iain Robert Dunning
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a plurality of actor computing units and a plurality of learner computing units. The actor computing units generate experience tuple trajectories that are used by the learner computing units to update learner action selection neural network parameters using a reinforcement learning technique. The reinforcement learning technique may be an off-policy actor critic reinforcement learning technique.
-
6.
公开(公告)号:US10936949B2
公开(公告)日:2021-03-02
申请号:US16508042
申请日:2019-07-10
Applicant: DeepMind Technologies Limited
Inventor: Marc Gendron-Bellemare , Jacob Lee Menick , Alexander Benjamin Graves , Koray Kavukcuoglu , Remi Munos
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model. In one aspect, a method includes receiving training data for training the machine learning model on a plurality of tasks, where each task includes multiple batches of training data. A task is selected in accordance with a current task selection policy. A batch of training data is selected from the selected task. The machine learning model is trained on the selected batch of training data to determine updated values of the model parameters. A learning progress measure that represents a progress of the training of the machine learning model as a result of training the machine learning model on the selected batch of training data is determined. The current task selection policy is updated using the learning progress measure.
-
7.
公开(公告)号:US12299574B2
公开(公告)日:2025-05-13
申请号:US18487428
申请日:2023-10-16
Applicant: DeepMind Technologies Limited
Inventor: Hubert Josef Soyer , Lasse Espeholt , Karen Simonyan , Yotam Doron , Vlad Firoiu , Volodymyr Mnih , Koray Kavukcuoglu , Remi Munos , Thomas Ward , Timothy James Alexander Harley , Iain Robert Dunning
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a plurality of actor computing units and a plurality of learner computing units. The actor computing units generate experience tuple trajectories that are used by the learner computing units to update learner action selection neural network parameters using a reinforcement learning technique. The reinforcement learning technique may be an off-policy actor critic reinforcement learning technique.
-
8.
公开(公告)号:US20240127060A1
公开(公告)日:2024-04-18
申请号:US18487428
申请日:2023-10-16
Applicant: DeepMind Technologies Limited
Inventor: Hubert Josef Soyer , Lasse Espeholt , Karen Simonyan , Yotam Doron , Vlad Firoiu , Volodymyr Mnih , Koray Kavukcuoglu , Remi Munos , Thomas Ward , Timothy James Alexander Harley , Iain Robert Dunning
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a plurality of actor computing units and a plurality of learner computing units. The actor computing units generate experience tuple trajectories that are used by the learner computing units to update learner action selection neural network parameters using a reinforcement learning technique. The reinforcement learning technique may be an off-policy actor critic reinforcement learning technique.
-
9.
公开(公告)号:US20230153617A1
公开(公告)日:2023-05-18
申请号:US18149771
申请日:2023-01-04
Applicant: DeepMind Technologies Limited
Inventor: Hubert Josef Soyer , Lasse Espeholt , Karen Simonyan , Yotam Doron , Vlad Firoiu , Volodymyr Mnih , Koray Kavukcuoglu , Remi Munos , Thomas Ward , Timothy James Alexander Harley , Iain Robert Dunning
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a plurality of actor computing units and a plurality of learner computing units. The actor computing units generate experience tuple trajectories that are used by the learner computing units to update learner action selection neural network parameters using a reinforcement learning technique. The reinforcement learning technique may be an off-policy actor critic reinforcement learning technique.
-
10.
公开(公告)号:US20210034970A1
公开(公告)日:2021-02-04
申请号:US16767049
申请日:2019-02-05
Applicant: DeepMind Technologies Limited
Inventor: Hubert Josef Soyer , Lasse Espeholt , Karen Simonyan , Yotam Doron , Vlad Firoiu , Volodymyr Mnih , Koray Kavukcuoglu , Remi Munos , Thomas Ward , Timothy James Alexander Harley , Iain Robert Dunning
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a plurality of actor computing units and a plurality of learner computing units. The actor computing units generate experience tuple trajectories that are used by the learner computing units to update learner action selection neural network parameters using a reinforcement learning technique. The reinforcement learning technique may be an off-policy actor critic reinforcement learning technique.
-
-
-
-
-
-
-
-
-