公开(公告)号：US11615310B2

公开(公告)日：2023-03-28

申请号：US16302592

申请日：2017-05-19

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Misha Man Ray Denil ,  Tom Schaul ,  Marcin Andrychowicz ,  Joao Ferdinando Gomes de Freitas ,  Sergio Gomez Colmenarejo ,  Matthew William Hoffman ,  David Benjamin Pfau

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for training machine learning models. One method includes obtaining a machine learning model, wherein the machine learning model comprises one or more model parameters, and the machine learning model is trained using gradient descent techniques to optimize an objective function; determining an update rule for the model parameters using a recurrent neural network (RNN); and applying a determined update rule for a final time step in a sequence of multiple time steps to the model parameters.

公开(公告)号：US20230376771A1

公开(公告)日：2023-11-23

申请号：US18180754

申请日：2023-03-08

Applicant: DeepMind Technologies Limited

Inventor： Misha Man Ray Denil ,  Tom Schaul ,  Marcin Andrychowicz ,  Joao Ferdinando Gomes de Freitas ,  Sergio Gomez Colmenarejo ,  Matthew William Hoffman ,  David Benjamin Pfau

IPC: G06N3/084 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06N3/084 ,  G06N3/044 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for training machine learning models. One method includes obtaining a machine learning model, wherein the machine learning model comprises one or more model parameters, and the machine learning model is trained using gradient descent techniques to optimize an objective function; determining an update rule for the model parameters using a recurrent neural network (RNN); and applying a determined update rule for a final time step in a sequence of multiple time steps to the model parameters.

公开(公告)号：US12271823B2

公开(公告)日：2025-04-08

申请号：US18180754

申请日：2023-03-08

Applicant: DeepMind Technologies Limited

Inventor： Misha Man Ray Denil ,  Tom Schaul ,  Marcin Andrychowicz ,  Joao Ferdinando Gomes de Freitas ,  Sergio Gomez Colmenarejo ,  Matthew William Hoffman ,  David Benjamin Pfau

IPC: G06N3/084 ,  G06N3/044 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for training machine learning models. One method includes obtaining a machine learning model, wherein the machine learning model comprises one or more model parameters, and the machine learning model is trained using gradient descent techniques to optimize an objective function; determining an update rule for the model parameters using a recurrent neural network (RNN); and applying a determined update rule for a final time step in a sequence of multiple time steps to the model parameters.

公开(公告)号：US11010664B2

公开(公告)日：2021-05-18

申请号：US15396000

申请日：2016-12-30

Applicant: DeepMind Technologies Limited

Inventor： Karol Piotr Kurach ,  Marcin Andrychowicz

IPC: G06N3/08 ,  G06N3/04 ,  G06F3/06 ,  G06F12/128 ,  G06N3/063

Abstract: Systems, methods, devices, and other techniques are disclosed for using an augmented neural network system to generate a sequence of outputs from a sequence of inputs. An augmented neural network system can include a controller neural network, a hierarchical external memory, and a memory access subsystem. The controller neural network receives a neural network input at each of a series of time steps processes the neural network input to generate a memory key for the time step. The external memory includes a set of memory nodes arranged as a binary tree. To provide an interface between the controller neural network and the external memory, the system includes a memory access subsystem that is configured to, for each of the series of time steps, perform one or more operations to generate a respective output for the time step. The capacity of the neural network system to account for long-range dependencies in input sequences may be extended. Also, memory access efficiency may be increased by structuring the external memory as a binary tree.