公开(公告)号：US11328385B2

公开(公告)日：2022-05-10

申请号：US16848741

申请日：2020-04-14

申请人： Adobe Inc.

发明人： Julia Gong ,  Yannick Hold-Geoffroy ,  Jingwan Lu

IPC分类号： G06T3/00 ,  G06T7/00 ,  G06T11/60

摘要： Techniques and systems are provided for configuring neural networks to perform warping of an object represented in an image to create a caricature of the object. For instance, in response to obtaining an image of an object, a warped image generator generates a warping field using the image as input. The warping field is generated using a model trained with pairings of training images and known warped images using supervised learning techniques and one or more losses. The warped image generator determines, based on the warping field, a set of displacements associated with pixels of the input image. These displacements indicate pixel displacement directions for the pixels of the input image. These displacements are applied to the digital image to generate a warped image of the object.

公开(公告)号：US10964060B2

公开(公告)日：2021-03-30

申请号：US16675641

申请日：2019-11-06

申请人： ADOBE INC.

发明人： Kalyan K. Sunkavalli ,  Yannick Hold-Geoffroy ,  Sunil Hadap ,  Matthew David Fisher ,  Jonathan Eisenmann ,  Emiliano Gambaretto

IPC分类号： G06T7/80 ,  G06N3/08 ,  G06T7/00 ,  G06N3/04

摘要： Embodiments of the present invention provide systems, methods, and computer storage media directed to generating training image data for a convolutional neural network, encoding parameters into a convolutional neural network, and employing a convolutional neural network that estimates camera calibration parameters of a camera responsible for capturing a given digital image. A plurality of different digital images can be extracted from a single panoramic image given a range of camera calibration parameters that correspond to a determined range of plausible camera calibration parameters. With each digital image in the plurality of extracted different digital images having a corresponding set of known camera calibration parameters, the digital images can be provided to the convolutional neural network to establish high-confidence correlations between detectable characteristics of a digital image and its corresponding set of camera calibration parameters. Once trained, the convolutional neural network can receive a new digital image, and based on detected image characteristics thereof, estimate a corresponding set of camera calibration parameters with a calculated level of confidence.

公开(公告)号：US20210065440A1

公开(公告)日：2021-03-04

申请号：US16558975

申请日：2019-09-03

申请人： Adobe Inc. ,  Université Laval

发明人： Kalyan Sunkavalli ,  Yannick Hold-Geoffroy ,  Christian Gagne ,  Marc-Andre Gardner ,  Jean-Francois Lalonde

IPC分类号： G06T15/50 ,  G06T7/50 ,  G06T7/90 ,  G06T7/70 ,  G06T7/60 ,  G06N3/08

摘要： This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional (“3D”) lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.

公开(公告)号：US20240273813A1

公开(公告)日：2024-08-15

申请号：US18168995

申请日：2023-02-14

申请人： Adobe Inc.

发明人： Jianming Zhang ,  Yichen Sheng ,  Julien Philip ,  Yannick Hold-Geoffroy ,  Xin Sun ,  He Zhang

IPC分类号： G06T15/60 ,  G06T7/60 ,  G06V10/60 ,  G06V10/74 ,  G06V10/82

CPC分类号： G06T15/60 ,  G06T7/60 ,  G06V10/60 ,  G06V10/761 ,  G06V10/82

摘要： The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object shadows for digital images utilizing corresponding geometry-aware buffer channels. For instance, in one or more embodiments, the disclosed systems generate, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image. The disclosed systems also generate, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map. Further, the disclosed systems modify the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

公开(公告)号：US11663775B2

公开(公告)日：2023-05-30

申请号：US17233861

申请日：2021-04-19

申请人： ADOBE INC.

发明人： Akshat Dave ,  Kalyan Krishna Sunkavalli ,  Yannick Hold-Geoffroy ,  Milos Hasan

IPC分类号： G06T15/50 ,  G06T15/00 ,  G06N3/08 ,  G06T15/04

CPC分类号： G06T15/506 ,  G06N3/08 ,  G06T15/005 ,  G06T15/04

摘要： Methods, system, and computer storage media are provided for generating physical-based materials for rendering digital objects with an appearance of a real-world material. Images depicted the real-world material, including diffuse component images and specular component images, are captured using different lighting patterns, which may include area lights. From the captured images, approximations of one or more material maps are determined using a photometric stereo technique. Based on the approximations and the captured images, a neural network system generates a set of material maps, such as a diffuse albedo material map, a normal material map, a specular albedo material map, and a roughness material map. The material maps from the neural network may be optimized based on a comparison of the input images of the real-world material and images rendered from the material maps.

公开(公告)号：US20230140146A1

公开(公告)日：2023-05-04

申请号：US17519117

申请日：2021-11-04

申请人： Adobe Inc.

发明人： Daichi Ito ,  Yijun Li ,  Yannick Hold-Geoffroy ,  Koki Madono ,  Jose Ignacio Echevarria Vallespi ,  Cameron Younger Smith

IPC分类号： G06T11/00 ,  G06K9/00 ,  G06T7/10

摘要： A vectorized caricature avatar generator receives a user image from which face parameters are generated. Segments of the user image including certain facial features (e.g., hair, facial hair, eyeglasses) are also identified. Segment parameter values are also determined, the segment parameter values being those parameter values from a set of caricature avatars that correspond to the segments of the user image. The face parameter values and the segment parameter values are used to generate a caricature avatar of the user in the user image.

公开(公告)号：US11568642B2

公开(公告)日：2023-01-31

申请号：US17068429

申请日：2020-10-12

申请人： ADOBE INC.

发明人： Michal Lukác ,  Oliver Wang ,  Jan Brejcha ,  Yannick Hold-Geoffroy ,  Martin {hacek over (C)}adík

IPC分类号： G06V20/20 ,  G06N20/00 ,  G06T7/70

摘要： Methods and systems are provided for facilitating large-scale augmented reality in relation to outdoor scenes using estimated camera pose information. In particular, camera pose information for an image can be estimated by matching the image to a rendered ground-truth terrain model with known camera pose information. To match images with such renders, data driven cross-domain feature embedding can be learned using a neural network. Cross-domain feature descriptors can be used for efficient and accurate feature matching between the image and the terrain model renders. This feature matching allows images to be localized in relation to the terrain model, which has known camera pose information. This known camera pose information can then be used to estimate camera pose information in relation to the image.

公开(公告)号：US20220335682A1

公开(公告)日：2022-10-20

申请号：US17233861

申请日：2021-04-19

申请人： ADOBE INC.

发明人： Akshat Dave ,  Kalyan Krishna Sunkavalli ,  Yannick Hold-Geoffroy ,  Milos Hasan

IPC分类号： G06T15/50 ,  G06T15/04 ,  G06N3/08 ,  G06T15/00

摘要： Methods, system, and computer storage media are provided for generating physical-based materials for rendering digital objects with an appearance of a real-world material. Images depicted the real-world material, including diffuse component images and specular component images, are captured using different lighting patterns, which may include area lights. From the captured images, approximations of one or more material maps are determined using a photometric stereo technique. Based on the approximations and the captured images, a neural network system generates a set of material maps, such as a diffuse albedo material map, a normal material map, a specular albedo material map, and a roughness material map. The material maps from the neural network may be optimized based on a comparison of the input images of the real-world material and images rendered from the material maps.

公开(公告)号：US20210264207A1

公开(公告)日：2021-08-26

申请号：US16802243

申请日：2020-02-26

申请人： ADOBE INC.

发明人： Cameron Smith ,  Yannick Hold-Geoffroy ,  Mariia Drozdova

IPC分类号： G06K9/62 ,  G06K9/46 ,  G06K9/00

摘要： Images can be edited to include features similar to a different target image. An unconditional generative adversarial network (GAN) is employed to edit features of an initial image based on a constraint determined from a target image. The constraint used by the GAN is determined from keypoints or segmentation masks of the target image, and edits are made to features of the initial image based on keypoints or segmentation masks of the initial image corresponding to those of the constraint from the target image. The GAN modifies the initial image based on a loss function having a variable for the constraint. The result of this optimization process is a modified initial image having features similar to the target image subject to the constraint determined from the identified keypoints or segmentation masks.

公开(公告)号：US20200074682A1

公开(公告)日：2020-03-05

申请号：US16675641

申请日：2019-11-06

申请人： ADOBE INC.

发明人： Kalyan K. Sunkavalli ,  Yannick Hold-Geoffroy ,  Sunil Hadap ,  Matthew David Fisher ,  Jonathan Eisenmann ,  Emiliano Gambaretto

IPC分类号： G06T7/80 ,  G06N3/08 ,  G06T7/00 ,  G06N3/04

摘要： Embodiments of the present invention provide systems, methods, and computer storage media directed to generating training image data for a convolutional neural network, encoding parameters into a convolutional neural network, and employing a convolutional neural network that estimates camera calibration parameters of a camera responsible for capturing a given digital image. A plurality of different digital images can be extracted from a single panoramic image given a range of camera calibration parameters that correspond to a determined range of plausible camera calibration parameters. With each digital image in the plurality of extracted different digital images having a corresponding set of known camera calibration parameters, the digital images can be provided to the convolutional neural network to establish high-confidence correlations between detectable characteristics of a digital image and its corresponding set of camera calibration parameters. Once trained, the convolutional neural network can receive a new digital image, and based on detected image characteristics thereof, estimate a corresponding set of camera calibration parameters with a calculated level of confidence.