公开(公告)号：US20230021497A1

公开(公告)日：2023-01-26

申请号：US17959132

申请日：2022-10-03

Applicant: DeepMind Technologies Limited

Inventor： Aaron Gerard Antonius van den Oord ,  Nal Emmerich Kalchbrenner ,  Karen Simonyan

IPC: G06V30/194 ,  G06N3/04 ,  G06N3/08 ,  H04N19/52 ,  G06K9/62 ,  H04N19/50 ,  G06V10/56

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating images using neural networks. One of the methods includes generating the output image pixel by pixel from a sequence of pixels taken from the output image, comprising, for each pixel in the output image, generating a respective score distribution over a discrete set of possible color values for each of the plurality of color channels.

公开(公告)号：US11321542B2

公开(公告)日：2022-05-03

申请号：US16927267

申请日：2020-07-13

Applicant: DeepMind Technologies Limited

Inventor： Nal Emmerich Kalchbrenner ,  Karen Simonyan ,  Lasse Espeholt

IPC: G06F40/44 ,  G06F40/58 ,  G06N3/04 ,  G06N3/08 ,  G10L15/197

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for language modeling. In one aspect, a system comprises: a masked convolutional decoder neural network that comprises a plurality of masked convolutional neural network layers and is configured to generate a respective probability distribution over a set of possible target embeddings at each of a plurality of time steps; and a modeling engine that is configured to use the respective probability distribution generated by the decoder neural network at each of the plurality of time steps to estimate a probability that a string represented by the target embeddings corresponding to the plurality of time steps belongs to the natural language.

公开(公告)号：US20210304847A1

公开(公告)日：2021-09-30

申请号：US17266724

申请日：2019-09-16

Applicant: DeepMind Technologies Limited

Inventor： Andrew W. Senior ,  James Kirkpatrick ,  Laurent Sifre ,  Richard Andrew Evans ,  Hugo Penedones ,  Chongli Qin ,  Ruoxi Sun ,  Karen Simonyan ,  John Jumper

IPC: G16B40/20 ,  G06N3/04 ,  G16B15/20 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing protein structure prediction. In one aspect, a method comprises, at each of one or more iterations: determining an alternative predicted structure of a given protein defined by alternative values of structure parameters; processing, using a geometry neural network, a network input comprising: (i) a representation of a sequence of amino acid residues in the given protein, and (ii) the alternative values of the structure parameters, to generate an output characterizing an alternative geometry score that is an estimate of a similarity measure between the alternative predicted structure and the actual structure of the given protein.

公开(公告)号：US20210034970A1

公开(公告)日：2021-02-04

申请号：US16767049

申请日：2019-02-05

Applicant: DeepMind Technologies Limited

Inventor： Hubert Josef Soyer ,  Lasse Espeholt ,  Karen Simonyan ,  Yotam Doron ,  Vlad Firoiu ,  Volodymyr Mnih ,  Koray Kavukcuoglu ,  Remi Munos ,  Thomas Ward ,  Timothy James Alexander Harley ,  Iain Robert Dunning

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a plurality of actor computing units and a plurality of learner computing units. The actor computing units generate experience tuple trajectories that are used by the learner computing units to update learner action selection neural network parameters using a reinforcement learning technique. The reinforcement learning technique may be an off-policy actor critic reinforcement learning technique.

公开(公告)号：US20210012197A1

公开(公告)日：2021-01-14

申请号：US16955420

申请日：2019-02-11

Applicant: DeepMind Technologies Limited

Inventor： Karen Simonyan ,  Nal Emmerich Kalchbrenner ,  Erich Konrad Elsen

IPC: G06N3/08 ,  G06F7/523

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using neural networks having Receive an input vector contiguous sparsity patterns. One of the methods includes storing a first parameter matrix of a neural network having a contiguous sparsity pattern in storage associated with a computing device. The computing device performs an inference pass of the neural network to generate an output vector, including reading, from the storage associated with the computing device, one or more activation values from the input vector, reading, from the storage associated with the computing device, a block of non-zero parameter values, and multiplying each of the one or more activation values by one or more of the block of non-zero parameter values.

公开(公告)号：US10803884B2

公开(公告)日：2020-10-13

申请号：US16390549

申请日：2019-04-22

Applicant: DeepMind Technologies Limited

Inventor： Aaron Gerard Antonius van den Oord ,  Sander Etienne Lea Dieleman ,  Nal Emmerich Kalchbrenner ,  Karen Simonyan ,  Oriol Vinyals

IPC: G10L13/00 ,  G10L25/30 ,  G06N3/04 ,  G10L13/06

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of audio data that comprises a respective audio sample at each of a plurality of time steps. One of the methods includes, for each of the time steps: providing a current sequence of audio data as input to a convolutional subnetwork, wherein the current sequence comprises the respective audio sample at each time step that precedes the time step in the output sequence, and wherein the convolutional subnetwork is configured to process the current sequence of audio data to generate an alternative representation for the time step; and providing the alternative representation for the time step as input to an output layer, wherein the output layer is configured to: process the alternative representation to generate an output that defines a score distribution over a plurality of possible audio samples for the time step.

公开(公告)号：US10748029B2

公开(公告)日：2020-08-18

申请号：US16041567

申请日：2018-07-20

Applicant: DeepMind Technologies Limited

Inventor： Maxwell Elliot Jaderberg ,  Karen Simonyan ,  Andrew Zisserman ,  Koray Kavukcuoglu

IPC: G06K9/03 ,  G06K9/52 ,  G06N3/04 ,  G06K9/46 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using an image processing neural network system that includes a spatial transformer module. One of the methods includes receiving an input feature map derived from the one or more input images, and applying a spatial transformation to the input feature map to generate a transformed feature map, comprising: processing the input feature map to generate spatial transformation parameters for the spatial transformation, and sampling from the input feature map in accordance with the spatial transformation parameters to generate the transformed feature map.

公开(公告)号：US10354015B2

公开(公告)日：2019-07-16

申请号：US16032971

申请日：2018-07-11

Applicant: DeepMind Technologies Limited

Inventor： Nal Emmerich Kalchbrenner ,  Karen Simonyan ,  Lasse Espeholt

IPC: G06F17/28 ,  G06N3/04 ,  G06N3/08 ,  G10L15/197

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for neural machine translation. In one aspect, a system is configured to receive an input sequence of source embeddings representing a source sequence of words in a source natural language and to generate an output sequence of target embeddings representing a target sequence of words that is a translation of the source sequence into a target natural language, the system comprising: a dilated convolutional neural network configured to process the input sequence of source embeddings to generate an encoded representation of the source sequence, and a masked dilated convolutional neural network configured to process the encoded representation of the source sequence to generate the output sequence of target embeddings.

公开(公告)号：US10304477B2

公开(公告)日：2019-05-28

申请号：US16030742

申请日：2018-07-09

Applicant: DeepMind Technologies Limited

Inventor： Aaron Gerard Antonius van den Oord ,  Sander Etienne Lea Dieleman ,  Nal Emmerich Kalchbrenner ,  Karen Simonyan ,  Oriol Vinyals

IPC: G10L13/00 ,  G10L25/30 ,  G06N3/04 ,  G10L13/06

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of audio data that comprises a respective audio sample at each of a plurality of time steps. One of the methods includes, for each of the time steps: providing a current sequence of audio data as input to a convolutional subnetwork, wherein the current sequence comprises the respective audio sample at each time step that precedes the time step in the output sequence, and wherein the convolutional subnetwork is configured to process the current sequence of audio data to generate an alternative representation for the time step; and providing the alternative representation for the time step as input to an output layer, wherein the output layer is configured to: process the alternative representation to generate an output that defines a score distribution over a plurality of possible audio samples for the time step.

公开(公告)号：US20190108833A1

公开(公告)日：2019-04-11

申请号：US16209661

申请日：2018-12-04

Applicant: DeepMind Technologies Limited

Inventor： Aaron Gerard Antonius van den Oord ,  Sander Etienne Lea Dieleman ,  Nal Emmerich Kalchbrenner ,  Karen Simonyan ,  Oriol Vinyals ,  Lasse Espeholt

IPC: G10L15/16 ,  G06N3/04 ,  G06N3/08 ,  G10L15/22 ,  G10L15/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing speech recognition by generating a neural network output from an audio data input sequence, where the neural network output characterizes words spoken in the audio data input sequence. One of the methods includes, for each of the audio data inputs, providing a current audio data input sequence that comprises the audio data input and the audio data inputs preceding the audio data input in the audio data input sequence to a convolutional subnetwork comprising a plurality of dilated convolutional neural network layers, wherein the convolutional subnetwork is configured to, for each of the plurality of audio data inputs: receive the current audio data input sequence for the audio data input, and process the current audio data input sequence to generate an alternative representation for the audio data input.