公开(公告)号：US20170140269A1

公开(公告)日：2017-05-18

申请号：US15349894

申请日：2016-11-11

Applicant: Google Inc.

Inventor： Tom Schaul ,  John Quan ,  David Silver

IPC: G06N3/08

CPC classification number: G06N3/08 ,  G06N3/088 ,  Y04S10/54

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network used to select actions performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes maintaining a replay memory, where the replay memory stores pieces of experience data generated as a result of the reinforcement learning agent interacting with the environment. Each piece of experience data is associated with a respective expected learning progress measure that is a measure of an expected amount of progress made in the training of the neural network if the neural network is trained on the piece of experience data. The method further includes selecting a piece of experience data from the replay memory by prioritizing for selection pieces of experience data having relatively higher expected learning progress measures and training the neural network on the selected piece of experience data.

公开(公告)号：US20160292568A1

公开(公告)日：2016-10-06

申请号：US15091840

申请日：2016-04-06

Applicant: Google Inc.

Inventor： Tom Schaul ,  Daniel George Horgan ,  Karol Gregor ,  David Silver

IPC: G06N3/08 ,  G06N99/00

CPC classification number: G06N3/08 ,  G06N3/0454 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning using goals and observations. One of the methods includes receiving an observation characterizing a current state of the environment; receiving a goal characterizing a target state from a set of target states of the environment; processing the observation using an observation neural network to generate a numeric representation of the observation; processing the goal using a goal neural network to generate a numeric representation of the goal; combining the numeric representation of the observation and the numeric representation of the goal to generate a combined representation; processing the combined representation using an action score neural network to generate a respective score for each action in the predetermined set of actions; and selecting the action to be performed using the respective scores for the actions in the predetermined set of actions.

Abstract translation: 方法，系统和装置，包括在计算机存储介质上编码的计算机程序，用于使用目标和观测来加强学习。 其中一种方法包括接收表征当前环境状态的观测值; 从环境的一组目标状态接收表征目标状态的目标; 使用观察神经网络处理观测以产生观察的数字表示; 使用目标神经网络处理目标以生成目标的数字表示; 组合观察的数字表示和目标的数字表示以生成组合表示; 使用动作评分神经网络处理所述组合表示以针对所述预定动作组中的每个动作生成相应的分数; 以及使用预定动作集中的动作的各个分数来选择要执行的动作。