公开(公告)号：US20190354867A1

公开(公告)日：2019-11-21

申请号：US16417522

申请日：2019-05-20

Applicant: DeepMind Technologies Limited

Inventor： Wojciech Czarnecki ,  Siddhant Jayakumar

IPC: G06N3/08 ,  G06N3/04 ,  G06N20/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning using agent curricula. One of the methods includes maintaining data specifying plurality of candidate agent policy neural networks; initializing mixing data that assigns a respective weight to each of the candidate agent policy neural networks; training the candidate agent policy neural networks using a reinforcement learning technique to generate combined action selection policies that result in improved performance on a reinforcement learning task; and during the training, repeatedly adjusting the weights in the mixing data to favor higher-performing candidate agent policy neural networks.

公开(公告)号：US20240394540A1

公开(公告)日：2024-11-28

申请号：US18674367

申请日：2024-05-24

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Jonathan Schwarz ,  Razvan Pascanu ,  Raia Thais Hadsell ,  Wojciech Czarnecki ,  Yee Whye Teh ,  Jelena Luketina

IPC: G06N3/084 ,  G06F18/22 ,  G06N3/08 ,  G06N5/02 ,  G06N20/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for scalable continual learning using neural networks. One of the methods includes receiving new training data for a new machine learning task; training an active subnetwork on the new training data to determine trained values of the active network parameters from initial values of the active network parameters while holding current values of the knowledge parameters fixed; and training a knowledge subnetwork on the new training data to determine updated values of the knowledge parameters from the current values of the knowledge parameters by training the knowledge subnetwork to generate knowledge outputs for the new training inputs that match active outputs generated by the trained active subnetwork for the new training inputs.

公开(公告)号：US11983634B2

公开(公告)日：2024-05-14

申请号：US17486842

申请日：2021-09-27

Applicant: DeepMind Technologies Limited

Inventor： Razvan Pascanu ,  Raia Thais Hadsell ,  Victor Constant Bapst ,  Wojciech Czarnecki ,  James Kirkpatrick ,  Yee Whye Teh ,  Nicolas Manfred Otto Heess

IPC: G06N3/08 ,  G06N3/084 ,  G06N3/10 ,  G06N5/043

CPC classification number: G06N3/084 ,  G06N3/10 ,  G06N5/043

Abstract: A method is proposed for training a multitask computer system, such as a multitask neural network system. The system comprises a set of trainable workers and a shared module. The trainable workers and shared module are trained on a plurality of different tasks, such that each worker learns to perform a corresponding one of the tasks according to a respective task policy, and said shared policy network learns a multitask policy which represents common behavior for the tasks. The coordinated training is performed by optimizing an objective function comprising, for each task: a reward term indicative of an expected reward earned by a worker in performing the corresponding task according to the task policy; and at least one entropy term which regularizes the distribution of the task policy towards the distribution of the multitask policy.

公开(公告)号：US11842261B2

公开(公告)日：2023-12-12

申请号：US17121679

申请日：2020-12-14

Applicant: DeepMind Technologies Limited

Inventor： Iain Robert Dunning ,  Wojciech Czarnecki ,  Maxwell Elliot Jaderberg

IPC: G06N3/045 ,  G06N3/08 ,  G06N3/047 ,  G06F17/18

CPC classification number: G06N3/045 ,  G06F17/18 ,  G06N3/047 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning. One of the methods includes selecting an action to be performed by the agent using both a slow updating recurrent neural network and a fast updating recurrent neural network that receives a fast updating input that includes the hidden state of the slow updating recurrent neural network.

公开(公告)号：US11113605B2

公开(公告)日：2021-09-07

申请号：US16417522

申请日：2019-05-20

Applicant: DeepMind Technologies Limited

Inventor： Wojciech Czarnecki ,  Siddhant Jayakumar

IPC: G06N3/08 ,  G06N20/20 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning using agent curricula. One of the methods includes maintaining data specifying plurality of candidate agent policy neural networks; initializing mixing data that assigns a respective weight to each of the candidate agent policy neural networks; training the candidate agent policy neural networks using a reinforcement learning technique to generate combined action selection policies that result in improved performance on a reinforcement learning task; and during the training, repeatedly adjusting the weights in the mixing data to favor higher-performing candidate agent policy neural networks.

公开(公告)号：US20210097373A1

公开(公告)日：2021-04-01

申请号：US17121679

申请日：2020-12-14

Applicant: DeepMind Technologies Limited

Inventor： Iain Robert Dunning ,  Wojciech Czarnecki ,  Maxwell Elliot Jaderberg

IPC: G06N3/04 ,  G06F17/18 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning. One of the methods includes selecting an action to be performed by the agent using both a slow updating recurrent neural network and a fast updating recurrent neural network that receives a fast updating input that includes the hidden state of the slow updating recurrent neural network.

公开(公告)号：US20190258938A1

公开(公告)日：2019-08-22

申请号：US16403385

申请日：2019-05-03

Applicant: DeepMind Technologies Limited

Inventor： Volodymyr Mnih ,  Wojciech Czarnecki ,  Maxwell Elliot Jaderberg ,  Tom Schaul ,  David Silver ,  Koray Kavukcuoglu

IPC: G06N3/08 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a reinforcement learning system. The method includes: training an action selection policy neural network, and during the training of the action selection neural network, training one or more auxiliary control neural networks and a reward prediction neural network. Each of the auxiliary control neural networks is configured to receive a respective intermediate output generated by the action selection policy neural network and generate a policy output for a corresponding auxiliary control task. The reward prediction neural network is configured to receive one or more intermediate outputs generated by the action selection policy neural network and generate a corresponding predicted reward. Training each of the auxiliary control neural networks and the reward prediction neural network comprises adjusting values of the respective auxiliary control parameters, reward prediction parameters, and the action selection policy network parameters.

公开(公告)号：US20240220774A1

公开(公告)日：2024-07-04

申请号：US18536065

申请日：2023-12-11

Applicant: DeepMind Technologies Limited

Inventor： Iain Robert Dunning ,  Wojciech Czarnecki ,  Maxwell Elliot Jaderberg

IPC: G06N3/045 ,  G06F17/18 ,  G06N3/047 ,  G06N3/08

CPC classification number: G06N3/045 ,  G06F17/18 ,  G06N3/047 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning. One of the methods includes selecting an action to be performed by the agent using both a slow updating recurrent neural network and a fast updating recurrent neural network that receives a fast updating input that includes the hidden state of the slow updating recurrent neural network.

公开(公告)号：US20240144015A1

公开(公告)日：2024-05-02

申请号：US18386954

申请日：2023-11-03

Applicant: DeepMind Technologies Limited

Inventor： Volodymyr Mnih ,  Wojciech Czarnecki ,  Maxwell Elliot Jaderberg ,  Tom Schaul ,  David Silver ,  Koray Kavukcuoglu

IPC: G06N3/084 ,  G06N3/006 ,  G06N3/044 ,  G06N3/045 ,  G06N20/00

CPC classification number: G06N3/084 ,  G06N3/006 ,  G06N3/044 ,  G06N3/045 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a reinforcement learning system. The method includes: training an action selection policy neural network, and during the training of the action selection neural network, training one or more auxiliary control neural networks and a reward prediction neural network. Each of the auxiliary control neural networks is configured to receive a respective intermediate output generated by the action selection policy neural network and generate a policy output for a corresponding auxiliary control task. The reward prediction neural network is configured to receive one or more intermediate outputs generated by the action selection policy neural network and generate a corresponding predicted reward. Training each of the auxiliary control neural networks and the reward prediction neural network comprises adjusting values of the respective auxiliary control parameters, reward prediction parameters, and the action selection policy network parameters.

公开(公告)号：US20220083869A1

公开(公告)日：2022-03-17

申请号：US17486842

申请日：2021-09-27

Applicant: DeepMind Technologies Limited

Inventor： Razvan Pascanu ,  Raia Thais Hadsell ,  Victor Constant Bapst ,  Wojciech Czarnecki ,  James Kirkpatrick ,  Yee Whye Teh ,  Nicolas Manfred Otto Heess

IPC: G06N3/08 ,  G06N3/10 ,  G06N5/04

Abstract: A method is proposed for training a multitask computer system, such as a multitask neural network system. The system comprises a set of trainable workers and a shared module. The trainable workers and shared module are trained on a plurality of different tasks, such that each worker learns to perform a corresponding one of the tasks according to a respective task policy, and said shared policy network learns a multitask policy which represents common behavior for the tasks. The coordinated training is performed by optimizing an objective function comprising, for each task: a reward term indicative of an expected reward earned by a worker in performing the corresponding task according to the task policy; and at least one entropy term which regularizes the distribution of the task policy towards the distribution of the multitask policy.