公开(公告)号：WO2020234457A1

公开(公告)日：2020-11-26

申请号：PCT/EP2020/064286

申请日：2020-05-22

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： BANINO, Andrea ,  BLUNDELL, Charles ,  BADIA, Adrià Puigdomènech ,  KOSTER, Raphael ,  KUMARAN, Sudarshan

IPC: G06N3/04 ,  G06N3/08 ,  G06N7/00 ,  G06N3/00

Abstract: A neural network based memory system with external memory for storing representations of knowledge items. The memory can be used to retrieve indirectly related knowledge items by recirculating queries, and is useful for relational reasoning. Implementations of the system control how many times queries are recirculated, and hence the degree of relational reasoning, to minimize computation.

公开(公告)号：WO2018083672A1

公开(公告)日：2018-05-11

申请号：PCT/IB2017/056907

申请日：2017-11-04

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： VIOLA, Fabio ,  MIROWSKI, Piotr Wojciech ,  BANINO, Andrea ,  PASCANU, Razvan ,  SOYER, Hubert Josef ,  BALLARD, Andrew James ,  KUMARAN, Sudarshan ,  HADSELL, Raia Thais ,  SIFRE, Laurent ,  GOROSHIN, Rostislav ,  KAVUKCUOGLU, Koray ,  DENIL, Misha Man Ray

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a reinforcement learning system. In one aspect, a method of training an action selection policy neural network for use in selecting actions to be performed by an agent navigating through an environment to accomplish one or more goals comprises: receiving an observation image characterizing a current state of the environment; processing, using the action selection policy neural network, an input comprising the observation image to generate an action selection output; processing, using a geometry-prediction neural network, an intermediate output generated by the action selection policy neural network to predict a value of a feature of a geometry of the environment when in the current state; and backpropagating a gradient of a geometry-based auxiliary loss into the action selection policy neural network to determine a geometry-based auxiliary update for current values of the network parameters.

Abstract translation: 包括编码在计算机存储介质上的用于训练强化学习系统的计算机程序的方法，系统和装置。 在一个方面，一种训练动作选择策略神经网络的方法用于选择要通过在环境中导航以实现一个或多个目标的代理执行的动作，包括：接收表征环境的当前状态的观察图像; 使用动作选择策略神经网络处理包括观察图像的输入以生成动作选择输出; 使用几何预测神经网络处理由动作选择策略神经网络产生的中间输出以预测当处于当前状态时环境的几何特征的值; 以及将基于几何的辅助损失的梯度反向传播到动作选择策略神经网络中以确定针对网络参数的当前值的基于几何的辅助更新。

公开(公告)号：WO2019215269A1

公开(公告)日：2019-11-14

申请号：PCT/EP2019/061890

申请日：2019-05-09

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： BANINO, Andrea ,  KUMARAN, Sudarshan ,  HADSELL, Raia Thais ,  URIA-MARTINEZ, Benigno

IPC: G06N3/00 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a grid cell neural network and an action selection neural network. The grid cell network is configured to: receive an input comprising data characterizing a velocity of the agent; process the input to generate a grid cell representation; and process the grid cell representation to generate an estimate of a position of the agent in the environment; the action selection neural network is configured to: receive an input comprising a grid cell representation and an observation characterizing a state of the environment; and process the input to generate an action selection network output.