公开(公告)号：US12165380B2

公开(公告)日：2024-12-10

申请号：US17309206

申请日：2019-11-15

Applicant: GOOGLE LLC

Inventor： Chloe LeGendre ,  Wan-Chun Ma ,  Graham Fyffe ,  John Flynn ,  Jessica Busch ,  Paul Debevec

IPC: G06V10/60 ,  G06V10/44 ,  G06V10/75 ,  G06V20/40

Abstract: An example method, apparatus, and computer-readable storage medium are provided to predict high-dynamic range (HDR) lighting from low-dynamic range (LDR) background images. In an example implementation, a method may include receiving low-dynamic range (LDR) background images of scenes, each LDR background image captured with appearance of one or more reference objects with different reflectance properties; and training a lighting estimation model based at least on the received LDR background images to predict high-dynamic range (HDR) lighting based at least on the trained model. In another example implementation, a method may include capturing a low-dynamic range (LDR) background image of a scene from an LDR video captured by a camera of the electronic computing device; predicting high-dynamic range (HDR) lighting for the image, the predicting, using a trained model, based at least on the LDR background image; and rendering a virtual object based at least on the predicted HDR lighting.

公开(公告)号：US20210406581A1

公开(公告)日：2021-12-30

申请号：US17309206

申请日：2019-11-15

Applicant: GOOGLE LLC

Inventor： Chloe LeGendre ,  Wan-Chun Ma ,  Graham Fyffe ,  John Flynn ,  Jessica Busch ,  Paul Debevec

IPC: G06K9/46 ,  G06K9/00 ,  G06K9/62

Abstract: An example method, apparatus, and computer-readable storage medium are provided to predict high-dynamic range (HDR) lighting from low-dynamic range (LDR) background images. In an example implementation, a method may include receiving low-dynamic range (LDR) background images of scenes, each LDR background image captured with appearance of one or more reference objects with different reflectance properties; and training a lighting estimation model based at least on the received LDR background images to predict high-dynamic range (HDR) lighting based at least on the trained model. In another example implementation, a method may include capturing a low-dynamic range (LDR) background image of a scene from an LDR video captured by a camera of the electronic computing device; predicting high-dynamic range (HDR) lighting for the image, the predicting, using a trained model, based at least on the LDR background image; and rendering a virtual object based at least on the predicted HDR lighting.

公开(公告)号：US20210166437A1

公开(公告)日：2021-06-03

申请号：US15733376

申请日：2019-10-16

Applicant: Google LLC

Inventor： Chloe LeGendre ,  Laurent Charbonnel ,  Christina Tong ,  Konstantine Nicholas John Tsotsos ,  Wan-Chun Ma ,  Paul Debevec

IPC: G06T11/00 ,  G06T3/20 ,  G06T3/60 ,  G06T7/70

Abstract: Systems, methods, and computer program products are described that implement obtaining, at an electronic computing device and for at least one image of a scene rendered in an Augmented Reality (AR) environment, a scene lighting estimation captured at a first time period. The scene lighting estimation may include at least a first image measurement associated with the scene. The implementations may include determining, at the electronic computing device, a second image measurement associated with the scene at a second time period, determining a function of the first image measurement and the second image measurement. Based on the determined function, the implementations may also include triggering calculation of a partial lighting estimation update or triggering calculation of a full lighting estimation update and rendering, on a screen of the electronic computing device and for the scene, the scene using the partial lighting estimation update or the full lighting estimation update.

公开(公告)号：US11288844B2

公开(公告)日：2022-03-29

申请号：US15733376

申请日：2019-10-16

Applicant: Google LLC

Inventor： Chloe LeGendre ,  Laurent Charbonnel ,  Christina Tong ,  Konstantine Nicholas John Tsotsos ,  Wan-Chun Ma ,  Paul Debevec

IPC: G06T11/00 ,  G06T7/70 ,  G06T3/20 ,  G06T3/60

Abstract: Systems, methods, and computer program products are described that implement obtaining, at an electronic computing device and for at least one image of a scene rendered in an Augmented Reality (AR) environment, a scene lighting estimation captured at a first time period. The scene lighting estimation may include at least a first image measurement associated with the scene. The implementations may include determining, at the electronic computing device, a second image measurement associated with the scene at a second time period, determining a function of the first image measurement and the second image measurement. Based on the determined function, the implementations may also include triggering calculation of a partial lighting estimation update or triggering calculation of a full lighting estimation update and rendering, on a screen of the electronic computing device and for the scene, the scene using the partial lighting estimation update or the full lighting estimation update.

公开(公告)号：US20220027659A1

公开(公告)日：2022-01-27

申请号：US17309171

申请日：2020-09-21

Applicant: Google LLC

Inventor： Chloe LeGendre ,  Paul Debevec ,  Wan-Chun Ma ,  Rohit Pandey ,  Sean Ryan Francesco Fanello ,  Christina Tong

IPC: G06K9/46 ,  G06K9/62 ,  H04N5/235 ,  H04N5/232

Abstract: Techniques of estimating lighting from portraits includes generating a lighting estimate from a single image of a face based on a machine learning (ML) system using multiple bidirectional reflection distribution functions (BRDFs) as a loss function. In some implementations, the ML system is trained using images of faces formed with HDR illumination computed from LDR imagery. The technical solution includes training a lighting estimation model in a supervised manner using a dataset of portraits and their corresponding ground truth illumination.