公开(公告)号：US20240062060A1

公开(公告)日：2024-02-22

申请号：US18267363

申请日：2021-12-20

Applicant: DeepMind Technologies Limited

Inventor： Sergey Bartunov ,  Felix Axel Gimeno Gil ,  Ingrid Karin von Glehn ,  Pawel Lichocki ,  Ivan Lobov ,  Vinod Nair ,  Brendan Timothy O'Donoghue ,  Nicolas Sonnerat ,  Christian Tjandraatmadja ,  Pengming Wang

IPC: G06N3/08 ,  G06F17/11

CPC classification number: G06N3/08 ,  G06F17/11

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for solving mixed integer programs (MIPs) using neural networks. One of the methods includes obtaining data specifying parameters of a MIP; generating, from the parameters of the MIP, an input representation; processing the input representation using an encoder neural network to generate a respective embedding for each of the integer variables; generating a plurality of partial assignments by selecting a respective second, proper subset of the integer variables; and for each of the variables in the respective second subset, generating, using at least the respective embedding for the variable, a respective additional constraint on the value of the variable; generating, for each of the partial assignments, a corresponding candidate final assignment that assigns a respective value to each of the plurality of variables; and selecting, as a final assignment for the MIP, one of the candidate final assignments.

公开(公告)号：US12277487B2

公开(公告)日：2025-04-15

申请号：US17441463

申请日：2020-05-19

Applicant: DeepMind Technologies Limited

Inventor： Sergey Bartunov ,  Jack William Rae ,  Timothy Paul Lillicrap ,  Simon Osindero

IPC: G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing associative memory. In one aspect a system comprises an associative memory neural network to process an input to generate an output that defines an energy corresponding to the input. A reading subsystem retrieves stored information from the associative memory neural network. The reading subsystem performs operations including receiving a given, i.e. query, input and retrieving a data element from the associative memory neural network that is associated with the given input. The retrieving is performed by iteratively adjusting the given input using the associative memory neural network.

公开(公告)号：US11983617B2

公开(公告)日：2024-05-14

申请号：US17102318

申请日：2020-11-23

Applicant: DeepMind Technologies Limited

Inventor： Jack William Rae ,  Timothy Paul Lillicrap ,  Sergey Bartunov

IPC: G06N3/045 ,  G06F16/22 ,  G06N3/08

CPC classification number: G06N3/045 ,  G06F16/2272 ,  G06N3/08

Abstract: A system for compressed data storage using a neural network. The system comprises a memory comprising a plurality of memory locations configured to store data; a query neural network configured to process a representation of an input data item to generate a query; an immutable key data store comprising key data for indexing the plurality of memory locations; an addressing system configured to process the key data and the query to generate a weighting associated with the plurality of memory locations; a memory read system configured to generate output memory data from the memory based upon the generated weighting associated with the plurality of memory locations and the data stored at the plurality of memory locations; and a memory write system configured to write received write data to the memory based upon the generated weighting associated with the plurality of memory locations.

公开(公告)号：US20250053780A1

公开(公告)日：2025-02-13

申请号：US18662972

申请日：2024-05-13

Applicant: DeepMind Technologies Limited

Inventor： Jack William Rae ,  Timothy Paul Lillicrap ,  Sergey Bartunov

IPC: G06N3/045 ,  G06F16/22 ,  G06N3/08

Abstract: A system for compressed data storage using a neural network. The system comprises a memory comprising a plurality of memory locations configured to store data; a query neural network configured to process a representation of an input data item to generate a query; an immutable key data store comprising key data for indexing the plurality of memory locations; an addressing system configured to process the key data and the query to generate a weighting associated with the plurality of memory locations; a memory read system configured to generate output memory data from the memory based upon the generated weighting associated with the plurality of memory locations and the data stored at the plurality of memory locations; and a memory write system configured to write received write data to the memory based upon the generated weighting associated with the plurality of memory locations.

公开(公告)号：US20210150314A1

公开(公告)日：2021-05-20

申请号：US17102318

申请日：2020-11-23

Applicant: DeepMind Technologies Limited

Inventor： Jack William Rae ,  Timothy Paul Lillicrap ,  Sergey Bartunov

IPC: G06N3/04 ,  G06F16/22 ,  G06N3/08

Abstract: A system for compressed data storage using a neural network. The system comprises a memory comprising a plurality of memory locations configured to store data; a query neural network configured to process a representation of an input data item to generate a query; an immutable key data store comprising key data for indexing the plurality of memory locations; an addressing system configured to process the key data and the query to generate a weighting associated with the plurality of memory locations; a memory read system configured to generate output memory data from the memory based upon the generated weighting associated with the plurality of memory locations and the data stored at the plurality of memory locations; and a memory write system configured to write received write data to the memory based upon the generated weighting associated with the plurality of memory locations.

公开(公告)号：US20220180147A1

公开(公告)日：2022-06-09

申请号：US17441463

申请日：2020-05-19

Applicant: DeepMind Technologies Limited

Inventor： Sergey Bartunov ,  Jack William Rae ,  Timothy Paul Lillicrap ,  Simon Osindero

IPC: G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing associative memory. In one aspect a system comprises an associative memory neural network to process an input to generate an output that defines an energy corresponding to the input. A reading subsystem retrieves stored information from the associative memory neural network. The reading subsystem performs operations including receiving a given, i.e. query, input and retrieving a data element from the associative memory neural network that is associated with the given input. The retrieving is performed by iteratively adjusting the given input using the associative memory neural network.

公开(公告)号：US10885426B2

公开(公告)日：2021-01-05

申请号：US15396289

申请日：2016-12-30

Applicant: DeepMind Technologies Limited

Inventor： Adam Anthony Santoro ,  Daniel Pieter Wiestra ,  Timothy Paul Lillicrap ,  Sergey Bartunov ,  Ivo Danihelka

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G06F12/123

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a controller neural network that includes a Least Recently Used Access (LRUA) subsystem configured to: maintain a respective usage weight for each of a plurality of locations in the external memory, and for each of the plurality of time steps: generate a respective reading weight for each location using a read key, read data from the locations in accordance with the reading weights, generate a respective writing weight for each of the locations from a respective reading weight from a preceding time step and the respective usage weight for the location, write a write vector to the locations in accordance with the writing weights, and update the respective usage weight from the respective reading weight and the respective writing weight.