公开(公告)号：US20250028931A1

公开(公告)日：2025-01-23

申请号：US18780270

申请日：2024-07-22

Applicant: DeepMind Technologies Limited

Inventor： Charles Blundell ,  Oriol Vinyals

IPC: G06N3/044 ,  G06F18/21 ,  G06F18/22 ,  G06F18/2413 ,  G06N3/08

Abstract: Methods, systems, and apparatus for classifying a new example using a comparison set of comparison examples. One method includes maintaining a comparison set, the comparison set including comparison examples and a respective label vector for each of the comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, in which the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.

公开(公告)号：US20240232580A1

公开(公告)日：2024-07-11

申请号：US18284595

申请日：2022-05-27

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Andrew Coulter Jaegle ,  Jean-Baptiste Alayrac ,  Sebastian Borgeaud Dit Avocat ,  Catalin-Dumitru Ionescu ,  Carl Doersch ,  Fengning Ding ,  Oriol Vinyals ,  Olivier Jean Hénaff ,  Skanda Kumar Koppula ,  Daniel Zoran ,  Andrew Brock ,  Evan Gerard Shelhamer ,  Andrew Zisserman ,  Joao Carreira

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a network output using a neural network. In one aspect, a method comprises: obtaining: (i) a network input to a neural network, and (ii) a set of query embeddings; processing the network input using the neural network to generate a network output that comprises a respective dimension corresponding to each query embedding in the set of query embeddings, comprising: processing the network input using an encoder block of the neural network to generate a representation of the network input as a set of latent embeddings; and processing: (i) the set of latent embeddings, and (ii) the set of query embeddings, using a cross-attention block that generates each dimension of the network output by cross-attention of a corresponding query embedding over the set of latent embeddings.

公开(公告)号：US20240054328A1

公开(公告)日：2024-02-15

申请号：US18144810

申请日：2023-05-08

Applicant: DeepMind Technologies Limited

Inventor： Yujia Li ,  Christopher James Dyer ,  Oriol Vinyals

IPC: G06N3/047 ,  G06F16/901 ,  G06F17/18 ,  G06N3/08 ,  G06N3/045

CPC classification number: G06N3/047 ,  G06F16/9024 ,  G06F17/18 ,  G06N3/08 ,  G06N3/045

Abstract: There is described a neural network system for generating a graph, the graph comprising a set of nodes and edges. The system comprises one or more neural networks configured to represent a probability distribution over sequences of node generating decisions and/or edge generating decisions, and one or more computers configured to sample the probability distribution represented by the one or more neural networks to generate a graph.

公开(公告)号：US20220319527A1

公开(公告)日：2022-10-06

申请号：US17674752

申请日：2022-02-17

Applicant: DeepMind Technologies Limited

Inventor： Cristina Garbacea ,  Aaron Gerard Antonius van den Oord ,  Yazhe Li ,  Sze Chie Lim ,  Alejandro Luebs ,  Oriol Vinyals ,  Thomas Chadwick Walters

IPC: G10L19/16 ,  G06N3/08 ,  G10L19/032 ,  G10L25/30

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating discrete latent representations of input audio data. Only the discrete latent representation needs to be transmitted from an encoder system to a decoder system in order for the decoder system to be able to effectively to decode, i.e., reconstruct, the input audio data.

公开(公告)号：US11328183B2

公开(公告)日：2022-05-10

申请号：US17019919

申请日：2020-09-14

Applicant: DeepMind Technologies Limited

Inventor： Daniel Pieter Wierstra ,  Yujia Li ,  Razvan Pascanu ,  Peter William Battaglia ,  Theophane Guillaume Weber ,  Lars Buesing ,  David Paul Reichert ,  Arthur Clement Guez ,  Danilo Jimenez Rezende ,  Adrià Puigdomènech Badia ,  Oriol Vinyals ,  Nicolas Manfred Otto Heess ,  Sebastien Henri Andre Racaniere

IPC: G06K9/00 ,  G06K9/62 ,  G06K9/68 ,  G06N3/04 ,  G06N3/08

Abstract: A neural network system is proposed. The neural network can be trained by model-based reinforcement learning to select actions to be performed by an agent interacting with an environment, to perform a task in an attempt to achieve a specified result. The system may comprise at least one imagination core which receives a current observation characterizing a current state of the environment, and optionally historical observations, and which includes a model of the environment. The imagination core may be configured to output trajectory data in response to the current observation, and/or historical observations. The trajectory data comprising a sequence of future features of the environment imagined by the imagination core. The system may also include a rollout encoder to encode the features, and an output stage to receive data derived from the rollout embedding and to output action policy data for identifying an action based on the current observation.

公开(公告)号：US11257507B2

公开(公告)日：2022-02-22

申请号：US16746703

申请日：2020-01-17

Applicant: DeepMind Technologies Limited

Inventor： Cristina Garbacea ,  Aaron Gerard Antonius van den Oord ,  Yazhe Li ,  Sze Chie Lim ,  Alejandro Luebs ,  Oriol Vinyals ,  Thomas Chadwick Walters

IPC: G10L19/16 ,  G10L19/032 ,  G10L25/30 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating discrete latent representations of input audio data. Only the discrete latent representation needs to be transmitted from an encoder system to a decoder system in order for the decoder system to be able to effectively to decode, i.e., reconstruct, the input audio data.

公开(公告)号：US20210224578A1

公开(公告)日：2021-07-22

申请号：US17223523

申请日：2021-04-06

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Charles Blundell ,  Oriol Vinyals

IPC: G06K9/62 ,  G06N3/08

Abstract: Methods, systems, and apparatus for classifying a new example using a comparison set of comparison examples. One method includes maintaining a comparison set, the comparison set including comparison examples and a respective label vector for each of the comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, in which the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.

公开(公告)号：US20210004689A1

公开(公告)日：2021-01-07

申请号：US17024217

申请日：2020-09-17

Applicant: DeepMind Technologies Limited

Inventor： Meire Fortunato ,  Charles Blundell ,  Oriol Vinyals

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network. In one aspect, a method includes maintaining data specifying, for each of the network parameters, current values of a respective set of distribution parameters that define a posterior distribution over possible values for the network parameter. A respective current training value for each of the network parameters is determined from a respective temporary gradient value for the network parameter. The current values of the respective sets of distribution parameters for the network parameters are updated in accordance with the respective current training values for the network parameters. The trained values of the network parameters are determined based on the updated current values of the respective sets of distribution parameters.

公开(公告)号：US10803884B2

公开(公告)日：2020-10-13

申请号：US16390549

申请日：2019-04-22

Applicant: DeepMind Technologies Limited

Inventor： Aaron Gerard Antonius van den Oord ,  Sander Etienne Lea Dieleman ,  Nal Emmerich Kalchbrenner ,  Karen Simonyan ,  Oriol Vinyals

IPC: G10L13/00 ,  G10L25/30 ,  G06N3/04 ,  G10L13/06

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of audio data that comprises a respective audio sample at each of a plurality of time steps. One of the methods includes, for each of the time steps: providing a current sequence of audio data as input to a convolutional subnetwork, wherein the current sequence comprises the respective audio sample at each time step that precedes the time step in the output sequence, and wherein the convolutional subnetwork is configured to process the current sequence of audio data to generate an alternative representation for the time step; and providing the alternative representation for the time step as input to an output layer, wherein the output layer is configured to: process the alternative representation to generate an output that defines a score distribution over a plurality of possible audio samples for the time step.

公开(公告)号：US20200279151A1

公开(公告)日：2020-09-03

申请号：US16759525

申请日：2018-10-29

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Yujia Li ,  Christopher James Dyer ,  Oriol Vinyals

IPC: G06N3/04 ,  G06N3/08 ,  G06F16/901 ,  G06F17/18

Abstract: There is described a neural network system for generating a graph, the graph comprising a set of nodes and edges. The system comprises one or more neural networks configured to represent a probability distribution over sequences of node generating decisions and/or edge generating decisions, and one or more computers configured to sample the probability distribution represented by the one or more neural networks to generate a graph.