公开(公告)号：US20200005147A1

公开(公告)日：2020-01-02

申请号：US16565245

申请日：2019-09-09

Applicant: DeepMind Technologies Limited

Inventor： Edward Thomas Grefenstette ,  Karl Moritz Hermann ,  Mustafa Suleyman ,  Philip Blunsom

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from the neural network output for the time step as a system output for the time step; maintaining a current state of the external memory; determining, from the neural network output for the time step, memory state parameters for the time step; updating the current state of the external memory using the memory state parameters for the time step; reading data from the external memory in accordance with the updated state of the external memory; and combining the data read from the external memory with a system input for the next time step to generate the neural network input for the next time step.

公开(公告)号：US11593640B2

公开(公告)日：2023-02-28

申请号：US16565245

申请日：2019-09-09

Applicant: DeepMind Technologies Limited

Inventor： Edward Thomas Grefenstette ,  Karl Moritz Hermann ,  Mustafa Suleyman ,  Philip Blunsom

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063 ,  G06N3/082

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from the neural network output for the time step as a system output for the time step; maintaining a current state of the external memory; determining, from the neural network output for the time step, memory state parameters for the time step; updating the current state of the external memory using the memory state parameters for the time step; reading data from the external memory in accordance with the updated state of the external memory; and combining the data read from the external memory with a system input for the next time step to generate the neural network input for the next time step.

公开(公告)号：US11151443B2

公开(公告)日：2021-10-19

申请号：US15424685

申请日：2017-02-03

Applicant: DeepMind Technologies Limited

Inventor： Ivo Danihelka ,  Gregory Duncan Wayne ,  Fu-min Wang ,  Edward Thomas Grefenstette ,  Jack William Rae ,  Alexander Benjamin Graves ,  Timothy Paul Lillicrap ,  Timothy James Alexander Harley ,  Jonathan James Hunt

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a sparse memory access subsystem that is configured to perform operations comprising generating a sparse set of reading weights that includes a respective reading weight for each of the plurality of locations in the external memory using the read key, reading data from the plurality of locations in the external memory in accordance with the sparse set of reading weights, generating a set of writing weights that includes a respective writing weight for each of the plurality of locations in the external memory, and writing the write vector to the plurality of locations in the external memory in accordance with the writing weights.

公开(公告)号：US12099928B2

公开(公告)日：2024-09-24

申请号：US18174394

申请日：2023-02-24

Applicant: DeepMind Technologies Limited

Inventor： Edward Thomas Grefenstette ,  Karl Moritz Hermann ,  Mustafa Suleyman ,  Philip Blunsom

IPC: G06N3/08 ,  G06N3/044 ,  G06N3/063 ,  G06N3/082

CPC classification number: G06N3/08 ,  G06N3/044 ,  G06N3/063 ,  G06N3/082

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from the neural network output for the time step as a system output for the time step; maintaining a current state of the external memory; determining, from the neural network output for the time step, memory state parameters for the time step; updating the current state of the external memory using the memory state parameters for the time step; reading data from the external memory in accordance with the updated state of the external memory; and combining the data read from the external memory with a system input for the next time step to generate the neural network input for the next time step.

公开(公告)号：US10410119B2

公开(公告)日：2019-09-10

申请号：US15172068

申请日：2016-06-02

Applicant: DeepMind Technologies Limited

Inventor： Edward Thomas Grefenstette ,  Karl Moritz Hermann ,  Mustafa Suleyman ,  Philip Blunsom

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from the neural network output for the time step as a system output for the time step; maintaining a current state of the external memory; determining, from the neural network output for the time step, memory state parameters for the time step; updating the current state of the external memory using the memory state parameters for the time step; reading data from the external memory in accordance with the updated state of the external memory; and combining the data read from the external memory with a system input for the next time step to generate the neural network input for the next time step.