-
公开(公告)号:US20220335636A1
公开(公告)日:2022-10-20
申请号:US17231833
申请日:2021-04-15
Applicant: ADOBE INC.
Inventor: Sai Bi , Zexiang Xu , Kalyan Krishna Sunkavalli , Milos Hasan , Yannick Hold-Geoffroy , David Jay Kriegman , Ravi Ramamoorthi
Abstract: A scene reconstruction system renders images of a scene with high-quality geometry and appearance and supports view synthesis, relighting, and scene editing. Given a set of input images of a scene, the scene reconstruction system trains a network to learn a volume representation of the scene that includes separate geometry and reflectance parameters. Using the volume representation, the scene reconstruction system can render images of the scene under arbitrary viewing (view synthesis) and lighting (relighting) locations. Additionally, the scene reconstruction system can render images that change the reflectance of objects in the scene (scene editing).
-
公开(公告)号:US11328385B2
公开(公告)日:2022-05-10
申请号:US16848741
申请日:2020-04-14
Applicant: Adobe Inc.
Inventor: Julia Gong , Yannick Hold-Geoffroy , Jingwan Lu
Abstract: Techniques and systems are provided for configuring neural networks to perform warping of an object represented in an image to create a caricature of the object. For instance, in response to obtaining an image of an object, a warped image generator generates a warping field using the image as input. The warping field is generated using a model trained with pairings of training images and known warped images using supervised learning techniques and one or more losses. The warped image generator determines, based on the warping field, a set of displacements associated with pixels of the input image. These displacements indicate pixel displacement directions for the pixels of the input image. These displacements are applied to the digital image to generate a warped image of the object.
-
公开(公告)号:US10964060B2
公开(公告)日:2021-03-30
申请号:US16675641
申请日:2019-11-06
Applicant: ADOBE INC.
Inventor: Kalyan K. Sunkavalli , Yannick Hold-Geoffroy , Sunil Hadap , Matthew David Fisher , Jonathan Eisenmann , Emiliano Gambaretto
Abstract: Embodiments of the present invention provide systems, methods, and computer storage media directed to generating training image data for a convolutional neural network, encoding parameters into a convolutional neural network, and employing a convolutional neural network that estimates camera calibration parameters of a camera responsible for capturing a given digital image. A plurality of different digital images can be extracted from a single panoramic image given a range of camera calibration parameters that correspond to a determined range of plausible camera calibration parameters. With each digital image in the plurality of extracted different digital images having a corresponding set of known camera calibration parameters, the digital images can be provided to the convolutional neural network to establish high-confidence correlations between detectable characteristics of a digital image and its corresponding set of camera calibration parameters. Once trained, the convolutional neural network can receive a new digital image, and based on detected image characteristics thereof, estimate a corresponding set of camera calibration parameters with a calculated level of confidence.
-
14.
公开(公告)号:US20210065440A1
公开(公告)日:2021-03-04
申请号:US16558975
申请日:2019-09-03
Applicant: Adobe Inc. , Université Laval
Inventor: Kalyan Sunkavalli , Yannick Hold-Geoffroy , Christian Gagne , Marc-Andre Gardner , Jean-Francois Lalonde
Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional (“3D”) lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.
-
公开(公告)号:US12236640B2
公开(公告)日:2025-02-25
申请号:US17656796
申请日:2022-03-28
Applicant: ADOBE INC.
Inventor: Jianming Zhang , Linyi Jin , Kevin Matzen , Oliver Wang , Yannick Hold-Geoffroy
Abstract: Systems and methods for image dense field based view calibration are provided. In one embodiment, an input image is applied to a dense field machine learning model that generates a vertical vector dense field (VVF) and a latitude dense field (LDF) from the input image. The VVF comprises a vertical vector of a projected vanishing point direction for each of the pixels of the input image. The latitude dense field (LDF) comprises a projected latitude value for the pixels of the input image. A dense field map for the input image comprising the VVF and the LDF can be directly or indirectly used for a variety of image processing manipulations. The VVF and LDF can be optionally used to derive traditional camera calibration parameters from uncontrolled images that have undergone undocumented or unknown manipulations.
-
16.
公开(公告)号:US20240273813A1
公开(公告)日:2024-08-15
申请号:US18168995
申请日:2023-02-14
Applicant: Adobe Inc.
Inventor: Jianming Zhang , Yichen Sheng , Julien Philip , Yannick Hold-Geoffroy , Xin Sun , He Zhang
CPC classification number: G06T15/60 , G06T7/60 , G06V10/60 , G06V10/761 , G06V10/82
Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object shadows for digital images utilizing corresponding geometry-aware buffer channels. For instance, in one or more embodiments, the disclosed systems generate, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image. The disclosed systems also generate, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map. Further, the disclosed systems modify the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.
-
公开(公告)号:US11663775B2
公开(公告)日:2023-05-30
申请号:US17233861
申请日:2021-04-19
Applicant: ADOBE INC.
Inventor: Akshat Dave , Kalyan Krishna Sunkavalli , Yannick Hold-Geoffroy , Milos Hasan
CPC classification number: G06T15/506 , G06N3/08 , G06T15/005 , G06T15/04
Abstract: Methods, system, and computer storage media are provided for generating physical-based materials for rendering digital objects with an appearance of a real-world material. Images depicted the real-world material, including diffuse component images and specular component images, are captured using different lighting patterns, which may include area lights. From the captured images, approximations of one or more material maps are determined using a photometric stereo technique. Based on the approximations and the captured images, a neural network system generates a set of material maps, such as a diffuse albedo material map, a normal material map, a specular albedo material map, and a roughness material map. The material maps from the neural network may be optimized based on a comparison of the input images of the real-world material and images rendered from the material maps.
-
公开(公告)号:US20230140146A1
公开(公告)日:2023-05-04
申请号:US17519117
申请日:2021-11-04
Applicant: Adobe Inc.
Inventor: Daichi Ito , Yijun Li , Yannick Hold-Geoffroy , Koki Madono , Jose Ignacio Echevarria Vallespi , Cameron Younger Smith
Abstract: A vectorized caricature avatar generator receives a user image from which face parameters are generated. Segments of the user image including certain facial features (e.g., hair, facial hair, eyeglasses) are also identified. Segment parameter values are also determined, the segment parameter values being those parameter values from a set of caricature avatars that correspond to the segments of the user image. The face parameter values and the segment parameter values are used to generate a caricature avatar of the user in the user image.
-
19.
公开(公告)号:US11568642B2
公开(公告)日:2023-01-31
申请号:US17068429
申请日:2020-10-12
Applicant: ADOBE INC.
Inventor: Michal Lukác , Oliver Wang , Jan Brejcha , Yannick Hold-Geoffroy , Martin {hacek over (C)}adík
Abstract: Methods and systems are provided for facilitating large-scale augmented reality in relation to outdoor scenes using estimated camera pose information. In particular, camera pose information for an image can be estimated by matching the image to a rendered ground-truth terrain model with known camera pose information. To match images with such renders, data driven cross-domain feature embedding can be learned using a neural network. Cross-domain feature descriptors can be used for efficient and accurate feature matching between the image and the terrain model renders. This feature matching allows images to be localized in relation to the terrain model, which has known camera pose information. This known camera pose information can then be used to estimate camera pose information in relation to the image.
-
公开(公告)号:US20220335682A1
公开(公告)日:2022-10-20
申请号:US17233861
申请日:2021-04-19
Applicant: ADOBE INC.
Inventor: Akshat Dave , Kalyan Krishna Sunkavalli , Yannick Hold-Geoffroy , Milos Hasan
Abstract: Methods, system, and computer storage media are provided for generating physical-based materials for rendering digital objects with an appearance of a real-world material. Images depicted the real-world material, including diffuse component images and specular component images, are captured using different lighting patterns, which may include area lights. From the captured images, approximations of one or more material maps are determined using a photometric stereo technique. Based on the approximations and the captured images, a neural network system generates a set of material maps, such as a diffuse albedo material map, a normal material map, a specular albedo material map, and a roughness material map. The material maps from the neural network may be optimized based on a comparison of the input images of the real-world material and images rendered from the material maps.
-
-
-
-
-
-
-
-
-