公开(公告)号：US20230098115A1

公开(公告)日：2023-03-30

申请号：US18062460

申请日：2022-12-06

Applicant: Adobe Inc. ,  Université Laval

Inventor： Kalyan Sunkavalli ,  Yannick Hold-Geoffroy ,  Christian Gagne ,  Marc-Andre Gardner ,  Jean-Francois Lalonde

IPC: G06T15/50 ,  G06T7/90 ,  G06T7/60 ,  G06T7/70 ,  G06N3/08 ,  G06T7/50

Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional (“3D”) lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.

公开(公告)号：US10957026B1

公开(公告)日：2021-03-23

申请号：US16564398

申请日：2019-09-09

Applicant: ADOBE INC.

Inventor： Jinsong Zhang ,  Kalyan K. Sunkavalli ,  Yannick Hold-Geoffroy ,  Sunil Hadap ,  Jonathan Eisenmann ,  Jean-Francois Lalonde

IPC: G06K9/00 ,  G06T5/00

Abstract: Methods and systems are provided for determining high-dynamic range lighting parameters for input low-dynamic range images. A neural network system can be trained to estimate high-dynamic range lighting parameters for input low-dynamic range images. The high-dynamic range lighting parameters can be based on sky color, sky turbidity, sun color, sun shape, and sun position. Such input low-dynamic range images can be low-dynamic range panorama images or low-dynamic range standard images. Such a neural network system can apply the estimates high-dynamic range lighting parameters to objects added to the low-dynamic range images.

公开(公告)号：US10665011B1

公开(公告)日：2020-05-26

申请号：US16428482

申请日：2019-05-31

Applicant: Adobe Inc. ,  Université Laval

Inventor： Kalyan Sunkavalli ,  Sunil Hadap ,  Nathan Carr ,  Jean-Francois Lalonde ,  Mathieu Garon

IPC: G06T15/80 ,  G06N20/10 ,  G06T19/00 ,  G06N3/08 ,  G06T15/00

Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that use a local-lighting-estimation-neural network to render a virtual object in a digital scene by using a local-lighting-estimation-neural network to analyze both global and local features of the digital scene and generate location-specific-lighting parameters for a designated position within the digital scene. For example, the disclosed systems extract and combine such global and local features from a digital scene using global network layers and local network layers of the local-lighting-estimation-neural network. In certain implementations, the disclosed systems can generate location-specific-lighting parameters using a neural-network architecture that combines global and local feature vectors to spatially vary lighting for different positions within a digital scene.

公开(公告)号：US20210065440A1

公开(公告)日：2021-03-04

申请号：US16558975

申请日：2019-09-03

Applicant: Adobe Inc. ,  Université Laval

Inventor： Kalyan Sunkavalli ,  Yannick Hold-Geoffroy ,  Christian Gagne ,  Marc-Andre Gardner ,  Jean-Francois Lalonde

IPC: G06T15/50 ,  G06T7/50 ,  G06T7/90 ,  G06T7/70 ,  G06T7/60 ,  G06N3/08

Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional (“3D”) lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.

公开(公告)号：US12008710B2

公开(公告)日：2024-06-11

申请号：US18062460

申请日：2022-12-06

Applicant: Adobe Inc. ,  Université Laval

Inventor： Kalyan Sunkavalli ,  Yannick Hold-Geoffroy ,  Christian Gagne ,  Marc-Andre Gardner ,  Jean-Francois Lalonde

IPC: G06T15/00 ,  G06N3/08 ,  G06T7/50 ,  G06T7/60 ,  G06T7/70 ,  G06T7/90 ,  G06T15/50

CPC classification number: G06T15/506 ,  G06N3/08 ,  G06T7/50 ,  G06T7/60 ,  G06T7/70 ,  G06T7/90 ,  G06T2200/24 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional (“3D”) lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.

公开(公告)号：US11538216B2

公开(公告)日：2022-12-27

申请号：US16558975

申请日：2019-09-03

Applicant: Adobe Inc. ,  Université Laval

Inventor： Kalyan Sunkavalli ,  Yannick Hold-Geoffroy ,  Christian Gagne ,  Marc-Andre Gardner ,  Jean-Francois Lalonde

IPC: G06T15/00 ,  G06T15/50 ,  G06T7/50 ,  G06T7/90 ,  G06T7/70 ,  G06N3/08 ,  G06T7/60

Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional (“3D”) lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.

公开(公告)号：US20210073955A1

公开(公告)日：2021-03-11

申请号：US16564398

申请日：2019-09-09

Applicant: ADOBE INC.

Inventor： Jinsong Zhang ,  Kalyan K. Sunkavalli ,  Yannick Hold-Geoffroy ,  Sunil Hadap ,  Jonathan Eisenmann ,  Jean-Francois Lalonde

IPC: G06T5/00

Abstract: Methods and systems are provided for determining high-dynamic range lighting parameters for input low-dynamic range images. A neural network system can be trained to estimate high-dynamic range lighting parameters for input low-dynamic range images. The high-dynamic range lighting parameters can be based on sky color, sky turbidity, sun color, sun shape, and sun position. Such input low-dynamic range images can be low-dynamic range panorama images or low-dynamic range standard images. Such a neural network system can apply the estimates high-dynamic range lighting parameters to objects added to the low-dynamic range images.