公开(公告)号：US11481619B2

公开(公告)日：2022-10-25

申请号：US16507675

申请日：2019-07-10

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  Kevin Wampler ,  Kalyan Krishna Sunkavalli ,  Elya Shechtman ,  Siddhant Jain

IPC: G06F17/13 ,  G06N3/08 ,  G06N3/10

Abstract: Techniques for incorporating a black-box function into a neural network are described. For example, an image editing function may be the black-box function and may be wrapped into a layer of the neural network. A set of parameters and a source image are provided to the black-box function, and the output image that represents the source image with the set of parameters applied to the source image is output from the black-box function. To address the issue that the black-box function may not be differentiable, a loss optimization may calculate the gradients of the function using, for example, a finite differences calculation, and the gradients are used to train the neural network to ensure the output image is representative of an expected ground truth image.

公开(公告)号：US11263259B2

公开(公告)日：2022-03-01

申请号：US16929429

申请日：2020-07-15

Applicant: Adobe Inc.

Inventor： Xiaohui Shen ,  Zhe Lin ,  Kalyan Krishna Sunkavalli ,  Hengshuang Zhao ,  Brian Lynn Price

IPC: G06F16/00 ,  G06F16/58 ,  G06N3/08 ,  G06F16/51

Abstract: Compositing aware digital image search techniques and systems are described that leverage machine learning. In one example, a compositing aware image search system employs a two-stream convolutional neural network (CNN) to jointly learn feature embeddings from foreground digital images that capture a foreground object and background digital images that capture a background scene. In order to train models of the convolutional neural networks, triplets of training digital images are used. Each triplet may include a positive foreground digital image and a positive background digital image taken from the same digital image. The triplet also contains a negative foreground or background digital image that is dissimilar to the positive foreground or background digital image that is also included as part of the triplet.

公开(公告)号：US10950038B2

公开(公告)日：2021-03-16

申请号：US16800783

申请日：2020-02-25

Applicant: ADOBE INC.

Inventor： Jeong Joon Park ,  Zhili Chen ,  Xin Sun ,  Vladimir Kim ,  Kalyan Krishna Sunkavalli ,  Duygu Ceylan Aksit

IPC: G06T15/50 ,  G06T19/00 ,  G06T15/60

Abstract: Matching an illumination of an embedded virtual object (VO) with current environment illumination conditions provides an enhanced immersive experience to a user. To match the VO and environment illuminations, illumination basis functions are determined based on preprocessing image data, captured as a first combination of intensities of direct illumination sources illuminates the environment. Each basis function corresponds to one of the direct illumination sources. During the capture of runtime image data, a second combination of intensities illuminates the environment. An illumination-weighting vector is determined based on the runtime image data. The determination of the weighting vector accounts for indirect illumination sources, such as surface reflections. The weighting vector encodes a superposition of the basis functions that corresponds to the second combination of intensities. The method illuminates the VO based on the weighting vector. The resulting illumination of the VO matches the second combination of the intensities and surface reflections.

公开(公告)号：US10747811B2

公开(公告)日：2020-08-18

申请号：US15986401

申请日：2018-05-22

Applicant: Adobe Inc.

Inventor： Xiaohui Shen ,  Zhe Lin ,  Kalyan Krishna Sunkavalli ,  Hengshuang Zhao ,  Brian Lynn Price

IPC: G06F16/00 ,  G06F16/58 ,  G06N3/08 ,  G06F16/51

Abstract: Compositing aware digital image search techniques and systems are described that leverage machine learning. In one example, a compositing aware image search system employs a two-stream convolutional neural network (CNN) to jointly learn feature embeddings from foreground digital images that capture a foreground object and background digital images that capture a background scene. In order to train models of the convolutional neural networks, triplets of training digital images are used. Each triplet may include a positive foreground digital image and a positive background digital image taken from the same digital image. The triplet also contains a negative foreground or background digital image that is dissimilar to the positive foreground or background digital image that is also included as part of the triplet.

公开(公告)号：US20200186714A1

公开(公告)日：2020-06-11

申请号：US16789195

申请日：2020-02-12

Applicant: Adobe Inc.

Inventor： Yannick Hold-Geoffroy ,  Sunil S. Hadap ,  Kalyan Krishna Sunkavalli ,  Emiliano Gambaretto

IPC: H04N5/232 ,  G06K9/00 ,  G06N3/04 ,  G06K9/62 ,  G06K9/46 ,  G06T5/00 ,  H04N5/235 ,  G06N3/08 ,  G06T15/50

Abstract: The present disclosure is directed toward systems and methods for predicting lighting conditions. In particular, the systems and methods described herein analyze a single low-dynamic range digital image to estimate a set of high-dynamic range lighting conditions associated with the single low-dynamic range lighting digital image. Additionally, the systems and methods described herein train a convolutional neural network to extrapolate lighting conditions from a digital image. The systems and methods also augment low-dynamic range information from the single low-dynamic range digital image by using a sky model algorithm to predict high-dynamic range lighting conditions.

公开(公告)号：US10609286B2

公开(公告)日：2020-03-31

申请号：US15621444

申请日：2017-06-13

Applicant: Adobe Inc.

Inventor： Yannick Hold-Geoffroy ,  Sunil S. Hadap ,  Kalyan Krishna Sunkavalli ,  Emiliano Gambaretto

IPC: G06T5/00 ,  H04N5/232 ,  G06K9/46 ,  G06T15/50 ,  G06N3/08 ,  H04N5/235 ,  G06K9/00 ,  G06K9/62 ,  G06N3/04

Abstract: The present disclosure is directed toward systems and methods for predicting lighting conditions. In particular, the systems and methods described herein analyze a single low-dynamic range digital image to estimate a set of high-dynamic range lighting conditions associated with the single low-dynamic range lighting digital image. Additionally, the systems and methods described herein train a convolutional neural network to extrapolate lighting conditions from a digital image. The systems and methods also augment low-dynamic range information from the single low-dynamic range digital image by using a sky model algorithm to predict high-dynamic range lighting conditions.

公开(公告)号：US11930303B2

公开(公告)日：2024-03-12

申请号：US17526998

申请日：2021-11-15

Applicant: Adobe Inc.

Inventor： Pulkit Gera ,  Oliver Wang ,  Kalyan Krishna Sunkavalli ,  Elya Shechtman ,  Chetan Nanda

IPC: H04N9/31 ,  H04N9/73

CPC classification number: H04N9/3182 ,  G06T5/92 ,  H04N9/73 ,  G06T2207/20081

Abstract: Systems and techniques for automatic digital parameter adjustment are described that leverage insights learned from an image set to automatically predict parameter values for an input item of digital visual content. To do so, the automatic digital parameter adjustment techniques described herein captures visual and contextual features of digital visual content to determine balanced visual output in a range of visual scenes and settings. The visual and contextual features of digital visual content are used to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. The parameter adjustment model exploits these feature interactions to determine visually pleasing parameter values for an input item of digital visual content. The predicted parameter values are output, allowing further adjustment to the parameter values.

公开(公告)号：US11663775B2

公开(公告)日：2023-05-30

申请号：US17233861

申请日：2021-04-19

Applicant: ADOBE INC.

Inventor： Akshat Dave ,  Kalyan Krishna Sunkavalli ,  Yannick Hold-Geoffroy ,  Milos Hasan

IPC: G06T15/50 ,  G06T15/00 ,  G06N3/08 ,  G06T15/04

CPC classification number: G06T15/506 ,  G06N3/08 ,  G06T15/005 ,  G06T15/04

Abstract: Methods, system, and computer storage media are provided for generating physical-based materials for rendering digital objects with an appearance of a real-world material. Images depicted the real-world material, including diffuse component images and specular component images, are captured using different lighting patterns, which may include area lights. From the captured images, approximations of one or more material maps are determined using a photometric stereo technique. Based on the approximations and the captured images, a neural network system generates a set of material maps, such as a diffuse albedo material map, a normal material map, a specular albedo material map, and a roughness material map. The material maps from the neural network may be optimized based on a comparison of the input images of the real-world material and images rendered from the material maps.

公开(公告)号：US20220335682A1

公开(公告)日：2022-10-20

申请号：US17233861

申请日：2021-04-19

Applicant: ADOBE INC.

Inventor： Akshat Dave ,  Kalyan Krishna Sunkavalli ,  Yannick Hold-Geoffroy ,  Milos Hasan

IPC: G06T15/50 ,  G06T15/04 ,  G06N3/08 ,  G06T15/00

Abstract: Methods, system, and computer storage media are provided for generating physical-based materials for rendering digital objects with an appearance of a real-world material. Images depicted the real-world material, including diffuse component images and specular component images, are captured using different lighting patterns, which may include area lights. From the captured images, approximations of one or more material maps are determined using a photometric stereo technique. Based on the approximations and the captured images, a neural network system generates a set of material maps, such as a diffuse albedo material map, a normal material map, a specular albedo material map, and a roughness material map. The material maps from the neural network may be optimized based on a comparison of the input images of the real-world material and images rendered from the material maps.

公开(公告)号：US20220182588A1

公开(公告)日：2022-06-09

申请号：US17526998

申请日：2021-11-15

Applicant: Adobe Inc.

Inventor： Pulkit Gera ,  Oliver Wang ,  Kalyan Krishna Sunkavalli ,  Elya Shechtman ,  Chetan Nanda

IPC: H04N9/31 ,  G06T5/00 ,  H04N9/73

Abstract: Systems and techniques for automatic digital parameter adjustment are described that leverage insights learned from an image set to automatically predict parameter values for an input item of digital visual content. To do so, the automatic digital parameter adjustment techniques described herein captures visual and contextual features of digital visual content to determine balanced visual output in a range of visual scenes and settings. The visual and contextual features of digital visual content are used to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. The parameter adjustment model exploits these feature interactions to determine visually pleasing parameter values for an input item of digital visual content. The predicted parameter values are output, allowing further adjustment to the parameter values.