公开(公告)号：US20240096001A1

公开(公告)日：2024-03-21

申请号：US18013983

申请日：2022-11-15

Applicant: Google LLC

Inventor： Seyed Mohammad Mehdi Sajjadi ,  Henning Meyer ,  Etienne François Régis Pot ,  Urs Michael Bergmann ,  Klaus Greff ,  Noha Radwan ,  Suhani Deepak-Ranu Vora ,  Mario Lu¢i¢ ,  Daniel Christopher Duckworth ,  Thomas Allen Funkhouser ,  Andrea Tagliasacchi

IPC: G06T15/20 ,  G06T15/06

CPC classification number: G06T15/20 ,  G06T15/06

Abstract: Provided are machine learning models that generate geometry-free neural scene representations through efficient object-centric novel-view synthesis. In particular, one example aspect of the present disclosure provides a novel framework in which an encoder model (e.g., an encoder transformer network) processes one or more RGB images (with or without pose) to produce a fully latent scene representation that can be passed to a decoder model (e.g., a decoder transformer network). Given one or more target poses, the decoder model can synthesize images in a single forward pass. In some example implementations, because transformers are used rather than convolutional or MLP networks, the encoder can learn an attention model that extracts enough 3D information about a scene from a small set of images to render novel views with correct projections, parallax, occlusions, and even semantics, without explicit geometry.

公开(公告)号：US20240119697A1

公开(公告)日：2024-04-11

申请号：US18012264

申请日：2022-10-10

Applicant: Google LLC

Inventor： Daniel Christopher Duckworth ,  Suhani Deepak-Ranu Vora ,  Noha Radwan ,  Klaus Greff ,  Henning Meyer ,  Kyle Adam Genova ,  Seyed Mohammad Mehdi Sajjadi ,  Etienne François Régis Pot ,  Andrea Tagliasacchi

IPC: G06V10/26 ,  G06T7/143 ,  G06T15/08

CPC classification number: G06V10/26 ,  G06T7/143 ,  G06T15/08 ,  G06T2207/20076 ,  G06T2207/20081

Abstract: Example embodiments of the present disclosure provide an example computer-implemented method for constructing a three-dimensional semantic segmentation of a scene from two-dimensional inputs. The example method includes obtaining, by a computing system comprising one or more processors, an image set comprising one or more views of a subject scene. The example method includes generating, by the computing system and based at least in part on the image set, a scene representation describing the subject scene in three dimensions. The example method includes generating, by the computing system and using a machine-learned semantic segmentation model framework, a multidimensional field of probability distributions over semantic categories, the multidimensional field defined over the three dimensions of the subject scene. The example method includes outputting, by the computing system, classification data for at least one location in the subject scene.