-
11.
公开(公告)号:US20200250528A1
公开(公告)日:2020-08-06
申请号:US16758461
申请日:2018-10-25
Applicant: DeepMind Technologies Limited
Inventor: Aaron Gerard Antonius van den Oord , Yutian Chen , Danilo Jimenez Rezende , Oriol Vinyals , Joao Ferdinando Gomes de Freitas , Scott Ellison Reed
IPC: G06N3/08
Abstract: A system comprising a causal convolutional neural network to autoregressively generate a succession of values of a data item conditioned upon previously generated values of the data item. The system includes support memory for a set of support data patches each of which comprises an encoding of an example data item. A soft attention mechanism attends to one or more patches when generating the current item value. The soft attention mechanism determines a set of scores for the support data patches, for example in the form of a soft attention query vector dependent upon the previously generated values of the data item. The soft attention query vector is used to query the memory. When generating the value of the data item at a current iteration layers of the causal convolutional neural network are conditioned upon the support data patches weighted by the scores.
-
公开(公告)号:US10304477B2
公开(公告)日:2019-05-28
申请号:US16030742
申请日:2018-07-09
Applicant: DeepMind Technologies Limited
Inventor: Aaron Gerard Antonius van den Oord , Sander Etienne Lea Dieleman , Nal Emmerich Kalchbrenner , Karen Simonyan , Oriol Vinyals
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of audio data that comprises a respective audio sample at each of a plurality of time steps. One of the methods includes, for each of the time steps: providing a current sequence of audio data as input to a convolutional subnetwork, wherein the current sequence comprises the respective audio sample at each time step that precedes the time step in the output sequence, and wherein the convolutional subnetwork is configured to process the current sequence of audio data to generate an alternative representation for the time step; and providing the alternative representation for the time step as input to an output layer, wherein the output layer is configured to: process the alternative representation to generate an output that defines a score distribution over a plurality of possible audio samples for the time step.
-
公开(公告)号:US20190108833A1
公开(公告)日:2019-04-11
申请号:US16209661
申请日:2018-12-04
Applicant: DeepMind Technologies Limited
Inventor: Aaron Gerard Antonius van den Oord , Sander Etienne Lea Dieleman , Nal Emmerich Kalchbrenner , Karen Simonyan , Oriol Vinyals , Lasse Espeholt
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing speech recognition by generating a neural network output from an audio data input sequence, where the neural network output characterizes words spoken in the audio data input sequence. One of the methods includes, for each of the audio data inputs, providing a current audio data input sequence that comprises the audio data input and the audio data inputs preceding the audio data input in the audio data input sequence to a convolutional subnetwork comprising a plurality of dilated convolutional neural network layers, wherein the convolutional subnetwork is configured to, for each of the plurality of audio data inputs: receive the current audio data input sequence for the audio data input, and process the current audio data input sequence to generate an alternative representation for the audio data input.
-
公开(公告)号:US20250103856A1
公开(公告)日:2025-03-27
申请号:US18832817
申请日:2023-01-30
Applicant: DeepMind Technologies Limited
Inventor: Joao Carreira , Andrew Coulter Jaegle , Skanda Kumar Koppula , Daniel Zoran , Adrià Recasens Continente , Catalin-Dumitru Ionescu , Olivier Jean Hénaff , Evan Gerard Shelhamer , Relja Arandjelovic , Matthew Botvinick , Oriol Vinyals , Karen Simonyan , Andrew Zisserman
IPC: G06N3/045
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for using a neural network to generate a network output that characterizes an entity. In one aspect, a method includes: obtaining a representation of the entity as a set of data element embeddings, obtaining a set of latent embeddings, and processing: (i) the set of data element embeddings, and (ii) the set of latent embeddings, using the neural network to generate the network output. The neural network includes a sequence of neural network blocks including: (i) one or more local cross-attention blocks, and (ii) an output block. Each local cross-attention block partitions the set of latent embeddings and the set of data element embeddings into proper subsets, and updates each proper subset of the set of latent embeddings using attention over only the corresponding proper subset of the set of data element embeddings.
-
公开(公告)号:US20240370725A1
公开(公告)日:2024-11-07
申请号:US18771770
申请日:2024-07-12
Applicant: DeepMind Technologies Limited
Inventor: David Silver , Oriol Vinyals , Maxwell Elliot Jaderberg
IPC: G06N3/08 , G06F18/214 , H04L9/40
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network having a plurality of policy parameters and used to select actions to be performed by an agent to control the agent to perform a particular task while interacting with one or more other agents in an environment. In one aspect, the method includes: maintaining data specifying a pool of candidate action selection policies; maintaining data specifying respective matchmaking policy; and training the policy neural network using a reinforcement learning technique to update the policy parameters. The policy parameters define policies to be used in controlling the agent to perform the particular task.
-
公开(公告)号:US12131248B2
公开(公告)日:2024-10-29
申请号:US18144810
申请日:2023-05-08
Applicant: DeepMind Technologies Limited
Inventor: Yujia Li , Christopher James Dyer , Oriol Vinyals
CPC classification number: G06N3/047 , G06F16/9024 , G06F17/18 , G06N3/045 , G06N3/08
Abstract: There is described a neural network system for generating a graph, the graph comprising a set of nodes and edges. The system comprises one or more neural networks configured to represent a probability distribution over sequences of node generating decisions and/or edge generating decisions, and one or more computers configured to sample the probability distribution represented by the one or more neural networks to generate a graph.
-
公开(公告)号:US12073304B2
公开(公告)日:2024-08-27
申请号:US18211085
申请日:2023-06-16
Applicant: DEEPMIND TECHNOLOGIES LIMITED
Inventor: Charles Blundell , Oriol Vinyals
IPC: G06N3/044 , G06F18/21 , G06F18/22 , G06F18/2413 , G06N3/08
CPC classification number: G06N3/044 , G06F18/217 , G06F18/22 , G06F18/2413 , G06N3/08
Abstract: Methods, systems, and apparatus for classifying a new example using a comparison set of comparison examples. One method includes maintaining a comparison set, the comparison set including comparison examples and a respective label vector for each of the comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, in which the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.
-
18.
公开(公告)号:US20240281654A1
公开(公告)日:2024-08-22
申请号:US18292165
申请日:2022-08-12
Applicant: DeepMind Technologies Limited
Inventor: Scott Ellison Reed , Konrad Zolna , Emilio Parisotto , Tom Erez , Alexander Novikov , Jack William Rae , Misha Man Ray Denil , Joao Ferdinando Gomes de Freitas , Oriol Vinyals , Sergio Gomez , Ashley Deloris Edwards , Jacob Bruce , Gabriel Barth-Maron
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent to interact with an environment using an action selection neural network. In one aspect, a method comprises, at each time step in a sequence of time steps: generating a current representation of a state of a task being performed by the agent in the environment as of the current time step as a sequence of data elements; autoregressively generating a sequence of data elements representing a current action to be performed by the agent at the current time step; and after autoregressively generating the sequence of data elements representing the current action, causing the agent to perform the current action at the current time step.
-
公开(公告)号:US12067491B2
公开(公告)日:2024-08-20
申请号:US18131567
申请日:2023-04-06
Applicant: DeepMind Technologies Limited
Inventor: David Silver , Oriol Vinyals , Maxwell Elliot Jaderberg
IPC: G06N20/00 , G06F18/214 , G06N3/08 , H04L9/40
CPC classification number: G06N3/08 , G06F18/214 , H04L63/205
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network having a plurality of policy parameters and used to select actions to be performed by an agent to control the agent to perform a particular task while interacting with one or more other agents in an environment. In one aspect, the method includes: maintaining data specifying a pool of candidate action selection policies; maintaining data specifying respective matchmaking policy; and training the policy neural network using a reinforcement learning technique to update the policy parameters. The policy parameters define policies to be used in controlling the agent to perform the particular task.
-
公开(公告)号:US20230334288A1
公开(公告)日:2023-10-19
申请号:US18211085
申请日:2023-06-16
Applicant: DEEPMIND TECHNOLOGIES LIMITED
Inventor: Charles Blundell , Oriol Vinyals
IPC: G06N3/044 , G06N3/08 , G06F18/2413 , G06F18/22 , G06F18/21
CPC classification number: G06N3/044 , G06N3/08 , G06F18/2413 , G06F18/22 , G06F18/217
Abstract: Methods, systems, and apparatus for classifying a new example using a comparison set of comparison examples. One method includes maintaining a comparison set, the comparison set including comparison examples and a respective label vector for each of the comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, in which the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.
-
-
-
-
-
-
-
-
-