公开(公告)号：US11756561B2

公开(公告)日：2023-09-12

申请号：US17674752

申请日：2022-02-17

Applicant: DeepMind Technologies Limited

Inventor： Cristina Garbacea ,  Aaron Gerard Antonius van den Oord ,  Yazhe Li ,  Sze Chie Lim ,  Alejandro Luebs ,  Oriol Vinyals ,  Thomas Chadwick Walters

IPC: G10L19/16 ,  G10L19/032 ,  G10L25/30 ,  G06N3/08

CPC classification number: G10L19/16 ,  G06N3/08 ,  G10L19/032 ,  G10L25/30

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating discrete latent representations of input audio data. Only the discrete latent representation needs to be transmitted from an encoder system to a decoder system in order for the decoder system to be able to effectively to decode, i.e., reconstruct, the input audio data.

公开(公告)号：US20230274125A1

公开(公告)日：2023-08-31

申请号：US18090243

申请日：2022-12-28

Applicant: DeepMind Technologies Limited

Inventor： Aaron Gerard Antonius van den Oord ,  Yazhe Li ,  Oriol Vinyals

IPC: G06N3/006 ,  G06F17/16 ,  G06N3/08 ,  G06F18/22 ,  G06N3/045 ,  G06N3/048 ,  G06V10/764 ,  G06V10/77 ,  G06V10/82

CPC classification number: G06N3/006 ,  G06F17/16 ,  G06N3/08 ,  G06F18/22 ,  G06N3/045 ,  G06N3/048 ,  G06V10/764 ,  G06V10/7715 ,  G06V10/82

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an encoder neural network that is configured to process an input observation to generate a latent representation of the input observation. In one aspect, a method includes: obtaining a sequence of observations; for each observation in the sequence of observations, processing the observation using the encoder neural network to generate a latent representation of the observation; for each of one or more given observations in the sequence of observations: generating a context latent representation of the given observation; and generating, from the context latent representation of the given observation, a respective estimate of the latent representations of one or more particular observations that are after the given observation in the sequence of observations.

公开(公告)号：US11715009B2

公开(公告)日：2023-08-01

申请号：US16303595

申请日：2017-05-19

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Oriol Vinyals ,  Alexander Benjamin Graves ,  Wojciech Czarnecki ,  Koray Kavukcuoglu ,  Simon Osindero ,  Maxwell Elliot Jaderberg

IPC: G06N3/084 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06N3/084 ,  G06N3/044 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network including a first subnetwork followed by a second subnetwork on training inputs by optimizing an objective function. In one aspect, a method includes processing a training input using the neural network to generate a training model output, including processing a subnetwork input for the training input using the first subnetwork to generate a subnetwork activation for the training input in accordance with current values of parameters of the first subnetwork, and providing the subnetwork activation as input to the second subnetwork; determining a synthetic gradient of the objective function for the first subnetwork by processing the subnetwork activation using a synthetic gradient model in accordance with current values of parameters of the synthetic gradient model; and updating the current values of the parameters of the first subnetwork using the synthetic gradient.

公开(公告)号：US11714993B2

公开(公告)日：2023-08-01

申请号：US17223523

申请日：2021-04-06

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Charles Blundell ,  Oriol Vinyals

IPC: G06N3/08 ,  G06N3/044 ,  G06F18/2413 ,  G06F18/22 ,  G06F18/21

CPC classification number: G06N3/044 ,  G06F18/217 ,  G06F18/22 ,  G06F18/2413 ,  G06N3/08

Abstract: Methods, systems, and apparatus for classifying a new example using a comparison set of comparison examples. One method includes maintaining a comparison set, the comparison set including comparison examples and a respective label vector for each of the comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, in which the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.

公开(公告)号：US20210383228A1

公开(公告)日：2021-12-09

申请号：US17338974

申请日：2021-06-04

Applicant: DeepMind Technologies Limited

Inventor： Petar Velickovic ,  Charles Blundell ,  Oriol Vinyals ,  Razvan Pascanu ,  Lars Buesing ,  Matthew Overlan

IPC: G06N3/08 ,  G06N3/04 ,  G06F16/23 ,  G06F16/901

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating prediction outputs characterizing a set of entities. In one aspect, a method comprises: obtaining data defining a graph, comprising: (i) a set of nodes, wherein each node represents a respective entity from the set of entities, (ii) a current set of edges, wherein each edge connects a pair of nodes, and (iii) a respective current embedding of each node; at each of a plurality of time steps: updating the respective current embedding of each node, comprising processing data defining the graph using a graph neural network; and updating the current set of edges based at least in part on the updated embeddings of the nodes; and at one or more of the plurality of time steps: generating a prediction output characterizing the set of entities based on the current embeddings of the nodes.

公开(公告)号：US11080591B2

公开(公告)日：2021-08-03

申请号：US15697407

申请日：2017-09-06

Applicant: DeepMind Technologies Limited

Inventor： Aaron Gerard Antonius van den Oord ,  Sander Etienne Lea Dieleman ,  Nal Emmerich Kalchbrenner ,  Karen Simonyan ,  Oriol Vinyals ,  Lasse Espeholt

IPC: G06N3/04 ,  G10L15/16 ,  G10L13/08 ,  G10L25/30 ,  G10L13/04 ,  G06N3/08 ,  G06F40/44 ,  G06F40/279 ,  G06F17/18

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing sequences using convolutional neural networks. One of the methods includes, for each of the time steps: providing a current sequence of audio data as input to a convolutional subnetwork, wherein the current sequence comprises the respective audio sample at each time step that precedes the time step in the output sequence, and wherein the convolutional subnetwork is configured to process the current sequence of audio data to generate an alternative representation for the time step; and providing the alternative representation for the time step as input to an output layer, wherein the output layer is configured to: process the alternative representation to generate an output that defines a score distribution over a plurality of possible audio samples for the time step.

公开(公告)号：US10997472B2

公开(公告)日：2021-05-04

申请号：US16303510

申请日：2017-05-19

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Charles Blundell ,  Oriol Vinyals

IPC: G06K9/62 ,  G06N3/08

Abstract: Methods, systems, and apparatus for classifying a new example using a comparison set of comparison examples. One method includes maintaining a comparison set, the comparison set including comparison examples and a respective label vector for each of the comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, in which the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.

公开(公告)号：US20210089834A1

公开(公告)日：2021-03-25

申请号：US17114324

申请日：2020-12-07

Applicant: DeepMind Technologies Limited

Inventor： Daniel Pieter Wierstra ,  Yujia Li ,  Razvan Pascanu ,  Peter William Battaglia ,  Theophane Guillaume Weber ,  Lars Buesing ,  David Paul Reichert ,  Oriol Vinyals ,  Nicolas Manfred Otto Heess ,  Sebastien Henri Andre Racaniere

IPC: G06K9/62 ,  G06K9/68 ,  G06N3/08 ,  G06N5/04

Abstract: A neural network system is proposed to select actions to be performed by an agent interacting with an environment to perform a task in an attempt to achieve a specified result. The system may include a controller to receive state data and context data, and to output action data. The system may also include an imagination module to receive the state and action data, and to output consequent state data. The system may also include a manager to receive the state data and the context data, and to output route data which defines whether the system is to execute an action or to imagine. The system may also include a memory to store the context data.

公开(公告)号：US20200234725A1

公开(公告)日：2020-07-23

申请号：US16746703

申请日：2020-01-17

Applicant: DeepMind Technologies Limited

Inventor： Cristina Garbacea ,  Aaron Gerard Antonius van den Oord ,  Yazhe Li ,  Sze Chie Lim ,  Alejandro Luebs ,  Oriol Vinyals ,  Thomas Chadwick Walters

IPC: G10L19/16 ,  G10L19/032 ,  G10L25/30 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating discrete latent representations of input audio data. Only the discrete latent representation needs to be transmitted from an encoder system to a decoder system in order for the decoder system to be able to effectively to decode, i.e., reconstruct, the input audio data.

公开(公告)号：US12242947B2

公开(公告)日：2025-03-04

申请号：US16759561

申请日：2018-10-29

Applicant: DeepMind Technologies Limited

Inventor： Pablo Sprechmann ,  Siddhant Jayakumar ,  Jack William Rae ,  Alexander Pritzel ,  Adrià Puigdomènech Badia ,  Oriol Vinyals ,  Razvan Pascanu ,  Charles Blundell

IPC: G06N3/045 ,  G06N3/02 ,  G06N3/04 ,  G06N3/044 ,  G06N3/084

Abstract: There is described herein a computer-implemented method of processing an input data item. The method comprises processing the input data item using a parametric model to generate output data, wherein the parametric model comprises a first sub-model and a second sub-model. The processing comprises processing, by the first sub-model, the input data to generate a query data item, retrieving, from a memory storing data point-value pairs, at least one data point-value pair based upon the query data item and modifying weights of the second sub-model based upon the retrieved at least one data point-value pair. The output data is then generated based upon the modified second sub-model.