公开(公告)号：US20220012898A1

公开(公告)日：2022-01-13

申请号：US17295321

申请日：2019-11-20

Applicant: DeepMind Technologies Limited

Inventor： Joao Carreira ,  Jean-Baptiste Alayrac ,  Andrew Zisserman

IPC: G06T7/215 ,  G06N3/04 ,  G06N3/08 ,  G06K9/62

Abstract: A computer-implemented neural network system for decomposing input video data. A video data input receives a sequence of video image frames. The sequence is encoded, using a 3D spatio-temporal encoder neural network, into a set of latent variables representing a compressed version of the sequence. A 3D spatio-temporal decoder neural network processes the set of latent variables to generate two or more sets of decomposed video data; these may be stored, communicated, and/or made available to a user interface. Input video including undesired features such as reflections, shadows, and occlusions may thus be decomposed into two or more video sequences, one in which the undesired features are suppressed, and another containing the undesired features.

公开(公告)号：US12067732B2

公开(公告)日：2024-08-20

申请号：US17295321

申请日：2019-11-20

Applicant: DeepMind Technologies Limited

Inventor： Joao Carreira ,  Jean-Baptiste Alayrac ,  Andrew Zisserman

IPC: G06T7/215 ,  G06F18/214 ,  G06N3/045 ,  G06N3/049 ,  G06N3/084

CPC classification number: G06T7/215 ,  G06F18/214 ,  G06N3/045 ,  G06N3/049 ,  G06N3/084 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A computer-implemented neural network system for decomposing input video data. A video data input receives a sequence of video image frames. The sequence is encoded, using a 3D spatio-temporal encoder neural network, into a set of latent variables representing a compressed version of the sequence. A 3D spatio-temporal decoder neural network processes the set of latent variables to generate two or more sets of decomposed video data; these may be stored, communicated, and/or made available to a user interface. Input video including undesired features such as reflections, shadows, and occlusions may thus be decomposed into two or more video sequences, one in which the undesired features are suppressed, and another containing the undesired features.

公开(公告)号：US20230350936A1

公开(公告)日：2023-11-02

申请号：US18141337

申请日：2023-04-28

Applicant: DeepMind Technologies Limited

Inventor： Jean-Baptiste Alayrac ,  Jeffrey Donahue ,  Karel Lenc ,  Karen Simonyan ,  Malcolm Kevin Campbell Reynolds ,  Pauline Luc ,  Arthur Mensch ,  Iain Barr ,  Antoine Miech ,  Yana Elizabeth Hasson ,  Katherine Elizabeth Millican ,  Roman Ring

IPC: G06F16/432 ,  G06F40/284 ,  G06F16/438

CPC classification number: G06F16/432 ,  G06F16/438 ,  G06F40/284

Abstract: A query processing system is described which receives a query input comprising an input token string and also at least one data item having a second, different modality, and generates a corresponding output token string.

公开(公告)号：US20240232580A1

公开(公告)日：2024-07-11

申请号：US18284595

申请日：2022-05-27

Applicant: DEEPMIND TECHNOLOGIES LIMITED

Inventor： Andrew Coulter Jaegle ,  Jean-Baptiste Alayrac ,  Sebastian Borgeaud Dit Avocat ,  Catalin-Dumitru Ionescu ,  Carl Doersch ,  Fengning Ding ,  Oriol Vinyals ,  Olivier Jean Hénaff ,  Skanda Kumar Koppula ,  Daniel Zoran ,  Andrew Brock ,  Evan Gerard Shelhamer ,  Andrew Zisserman ,  Joao Carreira

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a network output using a neural network. In one aspect, a method comprises: obtaining: (i) a network input to a neural network, and (ii) a set of query embeddings; processing the network input using the neural network to generate a network output that comprises a respective dimension corresponding to each query embedding in the set of query embeddings, comprising: processing the network input using an encoder block of the neural network to generate a representation of the network input as a set of latent embeddings; and processing: (i) the set of latent embeddings, and (ii) the set of query embeddings, using a cross-attention block that generates each dimension of the network output by cross-attention of a corresponding query embedding over the set of latent embeddings.

公开(公告)号：US20240104353A1

公开(公告)日：2024-03-28

申请号：US18274748

申请日：2022-02-08

Applicant: DeepMind Technologies Limited

Inventor： Rémi Bertrand Francis Leblond ,  Jean-Baptiste Alayrac ,  Laurent Sifre ,  Miruna Pîslar ,  Jean-Baptiste Lespiau ,  Ioannis Antonoglou ,  Karen Simonyan ,  David Silver ,  Oriol Vinyals

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: A computer-implemented method for generating an output token sequence from an input token sequence. The method combines a look ahead tree search, such as a Monte Carlo tree search, with a sequence-to-sequence neural network system. The sequence-to-sequence neural network system has a policy output defining a next token probability distribution, and may include a value neural network providing a value output to evaluate a sequence. An initial partial output sequence is extended using the look ahead tree search guided by the policy output and, in implementations, the value output, of the sequence-to-sequence neural network system until a complete output sequence is obtained.

公开(公告)号：US20230244907A1

公开(公告)日：2023-08-03

申请号：US18102985

申请日：2023-01-30

Applicant: DeepMind Technologies Limited

Inventor： Curtis Glenn-Macway Hawthorne ,  Andrew Coulter Jaegle ,  Catalina-Codruta Cangea ,  Sebastian Borgeaud Dit Avocat ,  Charlie Thomas Curtis Nash ,  Mateusz Malinowski ,  Sander Etienne Lea Dieleman ,  Oriol Vinyals ,  Matthew Botvinick ,  Ian Stuart Simon ,  Hannah Rachel Sheahan ,  Neil Zeghidour ,  Jean-Baptiste Alayrac ,  Joao Carreira ,  Jesse Engel

IPC: G06N3/044

CPC classification number: G06N3/044

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a sequence of data elements that includes a respective data element at each position in a sequence of positions. In one aspect, a method includes: for each position after a first position in the sequence of positions: obtaining a current sequence of data element embeddings that includes a respective data element embedding of each data element at a position that precedes the current position, obtaining a sequence of latent embeddings, and processing: (i) the current sequence of data element embeddings, and (ii) the sequence of latent embeddings, using a neural network to generate the data element at the current position. The neural network includes a sequence of neural network blocks including: (i) a cross-attention block, (ii) one or more self-attention blocks, and (iii) an output block.