公开(公告)号：US12106428B2

公开(公告)日：2024-10-01

申请号：US17686683

申请日：2022-03-04

Applicant: Google LLC

Inventor： Konstantinos Rematas ,  Thomas Allen Funkhouser ,  Vittorio Carlo Ferrari ,  Andrew Huaming Liu ,  Andrea Tagliasacchi ,  Pratul Preeti Srinivasan ,  Jonathan Tilton Barron

IPC: G06T15/20 ,  G06N20/00 ,  G06T5/50 ,  G06T5/92 ,  G06T17/10

CPC classification number: G06T15/205 ,  G06N20/00 ,  G06T5/50 ,  G06T5/92 ,  G06T17/10

Abstract: Systems and methods for view synthesis and three-dimensional reconstruction can learn an environment by utilizing a plurality of images of an environment and depth data. The use of depth data can be helpful when the quantity of images and different angles may be limited. For example, large outdoor environments can be difficult to learn due to the size, the varying image exposures, and the limited variance in view direction changes. The systems and methods can leverage a plurality of panoramic images and corresponding lidar data to accurately learn a large outdoor environment to then generate view synthesis outputs and three-dimensional reconstruction outputs. Training may include the use of an exposure correction network to address lighting exposure differences between training images.

公开(公告)号：US20230281913A1

公开(公告)日：2023-09-07

申请号：US17686683

申请日：2022-03-04

Applicant: Google LLC

Inventor： Konstantinos Rematas ,  Thomas Allen Funkhouser ,  Vittorio Carlo Ferrari ,  Andrew Huaming Liu ,  Andrea Tagliasacchi ,  Pratul Preeti Srinivasan ,  Jonathan Tilton Barron

IPC: G06T15/20 ,  G06T17/10 ,  G06N20/00 ,  G06T5/50 ,  G06T5/00

CPC classification number: G06T15/205 ,  G06T17/10 ,  G06N20/00 ,  G06T5/50 ,  G06T5/009

Abstract: Systems and methods for view synthesis and three-dimensional reconstruction can learn an environment by utilizing a plurality of images of an environment and depth data. The use of depth data can be helpful when the quantity of images and different angles may be limited. For example, large outdoor environments can be difficult to learn due to the size, the varying image exposures, and the limited variance in view direction changes. The systems and methods can leverage a plurality of panoramic images and corresponding lidar data to accurately learn a large outdoor environment to then generate view synthesis outputs and three-dimensional reconstruction outputs. Training may include the use of an exposure correction network to address lighting exposure differences between training images.

公开(公告)号：US20240420413A1

公开(公告)日：2024-12-19

申请号：US18813675

申请日：2024-08-23

Applicant: Google LLC

Inventor： Konstantinos Rematas ,  Thomas Allen Funkhouser ,  Vittorio Carlo Ferrari ,  Andrew Huaming Liu ,  Andrea Tagliasacchi ,  Pratul Preeti Srinivasan ,  Jonathan Tilton Barron

IPC: G06T15/20 ,  G06N20/00 ,  G06T5/50 ,  G06T5/92 ,  G06T17/10

Abstract: Systems and methods for view synthesis and three-dimensional reconstruction can learn an environment by utilizing a plurality of images of an environment and depth data. The use of depth data can be helpful when the quantity of images and different angles may be limited. For example, large outdoor environments can be difficult to learn due to the size, the varying image exposures, and the limited variance in view direction changes. The systems and methods can leverage a plurality of panoramic images and corresponding lidar data to accurately learn a large outdoor environment to then generate view synthesis outputs and three-dimensional reconstruction outputs. Training may include the use of an exposure correction network to address lighting exposure differences between training images.

公开(公告)号：US20240303908A1

公开(公告)日：2024-09-12

申请号：US18547628

申请日：2021-04-30

Applicant: GOOGLE LLC

Inventor： Yinda Zhang ,  Danhang Tang ,  Ruofei Du ,  Zhang Chen ,  Kyle Genova ,  Sofien Bouaziz ,  Thomas Allen Funkhouser ,  Sean Ryan Francesco Fanello ,  Christian Haene

IPC: G06T15/08 ,  G06T3/40 ,  G06T9/00 ,  G06T19/20

CPC classification number: G06T15/08 ,  G06T3/40 ,  G06T9/00 ,  G06T19/20

Abstract: A method including generating a first vector based on a first grid and a three-dimensional (3D) position associated with a first implicit representation (IR) of a 3D object, generating at least one second vector based on at least one second grid and an upsampled first grid, decoding the first vector to generate a second IR of the 3D object, decoding the at least one second vector to generate at least one third IR of the 3D object, generating a composite IR of the 3D object based on the second IR of the 3D object and the at least one third IR of the 3D object, and generating a reconstructed volume representing the 3D object based on the composite IR of the 3D object.

公开(公告)号：US20240096001A1

公开(公告)日：2024-03-21

申请号：US18013983

申请日：2022-11-15

Applicant: Google LLC

Inventor： Seyed Mohammad Mehdi Sajjadi ,  Henning Meyer ,  Etienne François Régis Pot ,  Urs Michael Bergmann ,  Klaus Greff ,  Noha Radwan ,  Suhani Deepak-Ranu Vora ,  Mario Lu¢i¢ ,  Daniel Christopher Duckworth ,  Thomas Allen Funkhouser ,  Andrea Tagliasacchi

IPC: G06T15/20 ,  G06T15/06

CPC classification number: G06T15/20 ,  G06T15/06

Abstract: Provided are machine learning models that generate geometry-free neural scene representations through efficient object-centric novel-view synthesis. In particular, one example aspect of the present disclosure provides a novel framework in which an encoder model (e.g., an encoder transformer network) processes one or more RGB images (with or without pose) to produce a fully latent scene representation that can be passed to a decoder model (e.g., a decoder transformer network). Given one or more target poses, the decoder model can synthesize images in a single forward pass. In some example implementations, because transformers are used rather than convolutional or MLP networks, the encoder can learn an attention model that extracts enough 3D information about a scene from a small set of images to render novel views with correct projections, parallax, occlusions, and even semantics, without explicit geometry.