-
公开(公告)号:US12165380B2
公开(公告)日:2024-12-10
申请号:US17309206
申请日:2019-11-15
Applicant: GOOGLE LLC
Inventor: Chloe LeGendre , Wan-Chun Ma , Graham Fyffe , John Flynn , Jessica Busch , Paul Debevec
Abstract: An example method, apparatus, and computer-readable storage medium are provided to predict high-dynamic range (HDR) lighting from low-dynamic range (LDR) background images. In an example implementation, a method may include receiving low-dynamic range (LDR) background images of scenes, each LDR background image captured with appearance of one or more reference objects with different reflectance properties; and training a lighting estimation model based at least on the received LDR background images to predict high-dynamic range (HDR) lighting based at least on the trained model. In another example implementation, a method may include capturing a low-dynamic range (LDR) background image of a scene from an LDR video captured by a camera of the electronic computing device; predicting high-dynamic range (HDR) lighting for the image, the predicting, using a trained model, based at least on the LDR background image; and rendering a virtual object based at least on the predicted HDR lighting.
-
公开(公告)号:US20210406581A1
公开(公告)日:2021-12-30
申请号:US17309206
申请日:2019-11-15
Applicant: GOOGLE LLC
Inventor: Chloe LeGendre , Wan-Chun Ma , Graham Fyffe , John Flynn , Jessica Busch , Paul Debevec
Abstract: An example method, apparatus, and computer-readable storage medium are provided to predict high-dynamic range (HDR) lighting from low-dynamic range (LDR) background images. In an example implementation, a method may include receiving low-dynamic range (LDR) background images of scenes, each LDR background image captured with appearance of one or more reference objects with different reflectance properties; and training a lighting estimation model based at least on the received LDR background images to predict high-dynamic range (HDR) lighting based at least on the trained model. In another example implementation, a method may include capturing a low-dynamic range (LDR) background image of a scene from an LDR video captured by a camera of the electronic computing device; predicting high-dynamic range (HDR) lighting for the image, the predicting, using a trained model, based at least on the LDR background image; and rendering a virtual object based at least on the predicted HDR lighting.
-
公开(公告)号:US20210368157A1
公开(公告)日:2021-11-25
申请号:US17325818
申请日:2021-05-20
Applicant: Google LLC
Inventor: Ryan Overbeck , Michael Joseph Broxton , John Flynn , Daniel William Erickson , Lars Peter Johannes Hedman , Matthew Nowicki DuVall , Jason Angelo Dourgarian , Jessica Lynn Busch , Matthew Stephen Whalen , Paul Debevec
IPC: H04N13/282 , H04N5/232 , H04N19/177 , H04N13/15 , H04N13/271 , H04N13/194 , H04N13/172 , H04N5/247 , H04N13/161
Abstract: Mechanisms for generating compressed images are provided. More particularly, methods, systems, and media for capturing, reconstructing, compressing, and rendering view-dependent immersive light field video with a layered mesh representation are provided.
-
公开(公告)号:US20230360182A1
公开(公告)日:2023-11-09
申请号:US18028930
申请日:2021-05-17
Applicant: Google LLC
Inventor: Sean Ryan Francesco Fanello , Yun-Ta Tsai , Rohit Kumar Pandey , Paul Debevec , Michael Milne , Chloe LeGendre , Jonathan Tilton Barron , Christoph Rhemann , Sofien Bouaziz , Navin Padman Sarma
CPC classification number: G06T5/009 , G06T7/60 , G06T7/70 , G06T15/506 , G06T2200/24 , G06T2207/20081 , G06T2207/20084 , G06T2207/20092 , G06T2207/20208 , G06T2207/30201
Abstract: Apparatus and methods related to applying lighting models to images of objects are provided. An example method includes applying a geometry model to an input image to determine a surface orientation map indicative of a distribution of lighting on an object based on a surface geometry. The method further includes applying an environmental light estimation model to the input image to determine a direction of synthetic lighting to be applied to the input image. The method also includes applying, based on the surface orientation map and the direction of synthetic lighting, a light energy model to determine a quotient image indicative of an amount of light energy to be applied to each pixel of the input image. The method additionally includes enhancing, based on the quotient image, a portion of the input image. One or more neural networks can be trained to perform one or more of the aforementioned aspects.
-
公开(公告)号:US20210166437A1
公开(公告)日:2021-06-03
申请号:US15733376
申请日:2019-10-16
Applicant: Google LLC
Inventor: Chloe LeGendre , Laurent Charbonnel , Christina Tong , Konstantine Nicholas John Tsotsos , Wan-Chun Ma , Paul Debevec
Abstract: Systems, methods, and computer program products are described that implement obtaining, at an electronic computing device and for at least one image of a scene rendered in an Augmented Reality (AR) environment, a scene lighting estimation captured at a first time period. The scene lighting estimation may include at least a first image measurement associated with the scene. The implementations may include determining, at the electronic computing device, a second image measurement associated with the scene at a second time period, determining a function of the first image measurement and the second image measurement. Based on the determined function, the implementations may also include triggering calculation of a partial lighting estimation update or triggering calculation of a full lighting estimation update and rendering, on a screen of the electronic computing device and for the scene, the scene using the partial lighting estimation update or the full lighting estimation update.
-
公开(公告)号:US11288844B2
公开(公告)日:2022-03-29
申请号:US15733376
申请日:2019-10-16
Applicant: Google LLC
Inventor: Chloe LeGendre , Laurent Charbonnel , Christina Tong , Konstantine Nicholas John Tsotsos , Wan-Chun Ma , Paul Debevec
Abstract: Systems, methods, and computer program products are described that implement obtaining, at an electronic computing device and for at least one image of a scene rendered in an Augmented Reality (AR) environment, a scene lighting estimation captured at a first time period. The scene lighting estimation may include at least a first image measurement associated with the scene. The implementations may include determining, at the electronic computing device, a second image measurement associated with the scene at a second time period, determining a function of the first image measurement and the second image measurement. Based on the determined function, the implementations may also include triggering calculation of a partial lighting estimation update or triggering calculation of a full lighting estimation update and rendering, on a screen of the electronic computing device and for the scene, the scene using the partial lighting estimation update or the full lighting estimation update.
-
公开(公告)号:US20220027659A1
公开(公告)日:2022-01-27
申请号:US17309171
申请日:2020-09-21
Applicant: Google LLC
Inventor: Chloe LeGendre , Paul Debevec , Wan-Chun Ma , Rohit Pandey , Sean Ryan Francesco Fanello , Christina Tong
Abstract: Techniques of estimating lighting from portraits includes generating a lighting estimate from a single image of a face based on a machine learning (ML) system using multiple bidirectional reflection distribution functions (BRDFs) as a loss function. In some implementations, the ML system is trained using images of faces formed with HDR illumination computed from LDR imagery. The technical solution includes training a lighting estimation model in a supervised manner using a dataset of portraits and their corresponding ground truth illumination.
-
8.
公开(公告)号:US20230206511A1
公开(公告)日:2023-06-29
申请号:US18117675
申请日:2023-03-06
Applicant: Google LLC
Inventor: Ryan Overbeck , Michael Joseph Broxton , John Flynn , Daniel William Erickson , Lars Peter Johannes Hedman , Matthew Nowicki DuVall , Jason Angelo Dourgarian , Jessica Lynn Busch , Matthew Stephen Whalen , Paul Debevec
CPC classification number: G06T9/001 , G06T17/20 , G06T15/04 , G06T7/50 , G06T2207/20021 , G06T2207/10024
Abstract: Mechanisms for generating compressed images are provided. More particularly, methods, systems, and media for capturing, reconstructing, compressing, and rendering view-dependent immersive light field video with a layered mesh representation are provided.
-
公开(公告)号:US11601636B2
公开(公告)日:2023-03-07
申请号:US17325818
申请日:2021-05-20
Applicant: Google LLC
Inventor: Ryan Overbeck , Michael Joseph Broxton , John Flynn , Daniel William Erickson , Lars Peter Johannes Hedman , Matthew Nowicki DuVall , Jason Angelo Dourgarian , Jessica Lynn Busch , Matthew Stephen Whalen , Paul Debevec
IPC: G06T15/08 , H04N13/282 , H04N5/232 , H04N19/177 , H04N13/15 , H04N13/161 , H04N13/194 , H04N13/172 , H04N5/247 , H04N13/271 , G06T15/20
Abstract: Mechanisms for generating compressed images are provided. More particularly, methods, systems, and media for capturing, reconstructing, compressing, and rendering view-dependent immersive light field video with a layered mesh representation are provided.
-
公开(公告)号:US10922878B2
公开(公告)日:2021-02-16
申请号:US16151276
申请日:2018-10-03
Applicant: GOOGLE LLC
Inventor: Ivan Neulander , Chloe LeGendre , Paul Debevec
IPC: G06T15/80 , G06T7/70 , G06T7/40 , G06T3/40 , G06T15/00 , G06T15/04 , G06T19/00 , G02B27/01 , G06T7/11 , H04N13/106 , H04N5/272 , G06T5/40 , G06T15/50
Abstract: Systems and methods for lighting inserted content are provided. For example, the inserted content may include augmented reality content that is inserted into an image of a physical space. An example system and method may include determining a location within an image to insert content. For example, the image may be captured by a camera device. The example system and method may also include identifying a region of the image based on the determined location to insert the content, determining at least one lighting parameter based on the identified region, and rendering the content using the determined at least one lighting parameter.
-
-
-
-
-
-
-
-
-