公开(公告)号：US20240169662A1

公开(公告)日：2024-05-23

申请号：US18517190

申请日：2023-11-22

Applicant: Google LLC

Inventor： Seyed Mohammad Mehdi Sajjadi ,  Klaus Greff ,  Etienne François Régis Pot ,  Daniel Christopher Duckworth ,  Mario Lucic ,  Aravindh Mahendran ,  Thomas Kipf

IPC: G06T15/20 ,  B25J9/16 ,  G06T7/73

CPC classification number: G06T15/205 ,  B25J9/1697 ,  G06T7/73 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: An example method includes obtaining, by a computing system, one or more source images of a scene; obtaining, by the computing system, a query associated with a target view of the scene, wherein at least a portion of the query is parameterized in a latent pose space; and generating, by the computing system and using a machine-learned image view synthesis model, an output image of the scene associated with the target view.

公开(公告)号：US20220383628A1

公开(公告)日：2022-12-01

申请号：US17726374

申请日：2022-04-21

Applicant: Google LLC

Inventor： Thomas Kipf ,  Gamaleldin Elsayed ,  Aravindh Mahendran ,  Austin Charles Stone ,  Sara Sabour Rouh Aghdam ,  Georg Heigold ,  Rico Jonschkowski ,  Alexey Dosovitskiy ,  Klaus Greff

IPC: G06V10/82 ,  G06V10/40 ,  G06V10/774

Abstract: A method includes obtaining first feature vectors and second feature vectors representing contents of a first and second image frame, respectively, of an input video. The method may also include generating, based on the first feature vectors, first slot vectors, where each slot vector represents attributes of a corresponding entity as represented in the first image frame, and generating, based on the first slot vectors, predicted slot vectors including a corresponding predicted slot vector that represents a transition of the attributes of the corresponding entity from the first to the second image frame. The method may additionally include generating, based on the predicted slot vectors and the second feature vectors, second slot vectors including a corresponding slot vector that represents the attributes of the corresponding entity as represented in the second image frame, and determining an output based on the predicted slot vectors or the second slot vectors.

公开(公告)号：US20220172066A1

公开(公告)日：2022-06-02

申请号：US17538891

申请日：2021-11-30

Applicant: Google LLC

Inventor： Thomas Unterthiner ,  Alexey Dosovitskiy ,  Aravindh Mahendran ,  Dirk Weissenborn ,  Jakob D. Uszkoreit ,  Jean-Baptiste Cordonnier

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62 ,  G06K9/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to process images. One of the methods includes obtaining a training image; processing the training image using a first subnetwork to generate, for each of a plurality of first image patches of the training image, a relevance score; generating, using the relevance scores, one or more second image patches of the training image by performing one or more differentiable operations on the relevance scores; processing the one or more second image patches using a second subnetwork to generate a prediction about the training image; determining an error of the training network output; and generating a parameter update for the first subnetwork, comprising backpropagating gradients determined according to the error of the training network output through i) the second subnetwork, ii) the one or more differentiable operations, and iii) the first subnetwork.

公开(公告)号：US20210383199A1

公开(公告)日：2021-12-09

申请号：US16927018

申请日：2020-07-13

Applicant: Google LLC

Inventor： Dirk Weissenborn ,  Jakob Uszkoreit ,  Thomas Unterthiner ,  Aravindh Mahendran ,  Francesco Locatello ,  Thomas Kipf ,  Georg Heigold ,  Alexey Dosovitskiy

IPC: G06N3/063 ,  G06N3/08 ,  G06F17/16

Abstract: A method involves receiving a perceptual representation including a plurality of feature vectors, and initializing a plurality of slot vectors represented by a neural network memory unit. Each respective slot vector is configured to represent a corresponding entity in the perceptual representation. The method also involves determining an attention matrix based on a product of the plurality of feature vectors transformed by a key function and the plurality of slot vectors transformed by a query function. Each respective value of a plurality of values along each respective dimension of the attention matrix is normalized with respect to the plurality of values. The method additionally involves determining an update matrix based on the plurality of feature vectors transformed by a value function and the attention matrix, and updating the plurality of slot vectors based on the update matrix by way of the neural network memory unit.