公开(公告)号：US20220383628A1

公开(公告)日：2022-12-01

申请号：US17726374

申请日：2022-04-21

Applicant: Google LLC

Inventor： Thomas Kipf ,  Gamaleldin Elsayed ,  Aravindh Mahendran ,  Austin Charles Stone ,  Sara Sabour Rouh Aghdam ,  Georg Heigold ,  Rico Jonschkowski ,  Alexey Dosovitskiy ,  Klaus Greff

IPC: G06V10/82 ,  G06V10/40 ,  G06V10/774

Abstract: A method includes obtaining first feature vectors and second feature vectors representing contents of a first and second image frame, respectively, of an input video. The method may also include generating, based on the first feature vectors, first slot vectors, where each slot vector represents attributes of a corresponding entity as represented in the first image frame, and generating, based on the first slot vectors, predicted slot vectors including a corresponding predicted slot vector that represents a transition of the attributes of the corresponding entity from the first to the second image frame. The method may additionally include generating, based on the predicted slot vectors and the second feature vectors, second slot vectors including a corresponding slot vector that represents the attributes of the corresponding entity as represented in the second image frame, and determining an output based on the predicted slot vectors or the second slot vectors.

公开(公告)号：US20240119697A1

公开(公告)日：2024-04-11

申请号：US18012264

申请日：2022-10-10

Applicant: Google LLC

Inventor： Daniel Christopher Duckworth ,  Suhani Deepak-Ranu Vora ,  Noha Radwan ,  Klaus Greff ,  Henning Meyer ,  Kyle Adam Genova ,  Seyed Mohammad Mehdi Sajjadi ,  Etienne François Régis Pot ,  Andrea Tagliasacchi

IPC: G06V10/26 ,  G06T7/143 ,  G06T15/08

CPC classification number: G06V10/26 ,  G06T7/143 ,  G06T15/08 ,  G06T2207/20076 ,  G06T2207/20081

Abstract: Example embodiments of the present disclosure provide an example computer-implemented method for constructing a three-dimensional semantic segmentation of a scene from two-dimensional inputs. The example method includes obtaining, by a computing system comprising one or more processors, an image set comprising one or more views of a subject scene. The example method includes generating, by the computing system and based at least in part on the image set, a scene representation describing the subject scene in three dimensions. The example method includes generating, by the computing system and using a machine-learned semantic segmentation model framework, a multidimensional field of probability distributions over semantic categories, the multidimensional field defined over the three dimensions of the subject scene. The example method includes outputting, by the computing system, classification data for at least one location in the subject scene.

公开(公告)号：US20240169662A1

公开(公告)日：2024-05-23

申请号：US18517190

申请日：2023-11-22

Applicant: Google LLC

Inventor： Seyed Mohammad Mehdi Sajjadi ,  Klaus Greff ,  Etienne François Régis Pot ,  Daniel Christopher Duckworth ,  Mario Lucic ,  Aravindh Mahendran ,  Thomas Kipf

IPC: G06T15/20 ,  B25J9/16 ,  G06T7/73

CPC classification number: G06T15/205 ,  B25J9/1697 ,  G06T7/73 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: An example method includes obtaining, by a computing system, one or more source images of a scene; obtaining, by the computing system, a query associated with a target view of the scene, wherein at least a portion of the query is parameterized in a latent pose space; and generating, by the computing system and using a machine-learned image view synthesis model, an output image of the scene associated with the target view.

公开(公告)号：US20240096001A1

公开(公告)日：2024-03-21

申请号：US18013983

申请日：2022-11-15

Applicant: Google LLC

Inventor： Seyed Mohammad Mehdi Sajjadi ,  Henning Meyer ,  Etienne François Régis Pot ,  Urs Michael Bergmann ,  Klaus Greff ,  Noha Radwan ,  Suhani Deepak-Ranu Vora ,  Mario Lu¢i¢ ,  Daniel Christopher Duckworth ,  Thomas Allen Funkhouser ,  Andrea Tagliasacchi

IPC: G06T15/20 ,  G06T15/06

CPC classification number: G06T15/20 ,  G06T15/06

Abstract: Provided are machine learning models that generate geometry-free neural scene representations through efficient object-centric novel-view synthesis. In particular, one example aspect of the present disclosure provides a novel framework in which an encoder model (e.g., an encoder transformer network) processes one or more RGB images (with or without pose) to produce a fully latent scene representation that can be passed to a decoder model (e.g., a decoder transformer network). Given one or more target poses, the decoder model can synthesize images in a single forward pass. In some example implementations, because transformers are used rather than convolutional or MLP networks, the encoder can learn an attention model that extracts enough 3D information about a scene from a small set of images to render novel views with correct projections, parallax, occlusions, and even semantics, without explicit geometry.