公开(公告)号：US20200090389A1

公开(公告)日：2020-03-19

申请号：US16676733

申请日：2019-11-07

Applicant: Adobe Inc.

Inventor： Sunil Hadap ,  Elya Shechtman ,  Zhixin Shu ,  Kalyan Sunkavalli ,  Mehmet Yumer

IPC: G06T11/60 ,  G06N3/04 ,  G06N3/08 ,  G06T5/00

Abstract: Techniques are disclosed for performing manipulation of facial images using an artificial neural network. A facial rendering and generation network and method learns one or more compact, meaningful manifolds of facial appearance, by disentanglement of a facial image into intrinsic facial properties, and enables facial edits by traversing paths of such manifold(s). The facial rendering and generation network is able to handle a much wider range of manipulations including changes to, for example, viewpoint, lighting, expression, and even higher-level attributes like facial hair and age—aspects that cannot be represented using previous models.

公开(公告)号：US20200074600A1

公开(公告)日：2020-03-05

申请号：US16678072

申请日：2019-11-08

Applicant: Adobe Inc.

Inventor： Kalyan Sunkavalli ,  Mehmet Ersin Yumer ,  Marc-Andre Gardner ,  Xiaohui Shen ,  Jonathan Eisenmann ,  Emiliano Gambaretto

IPC: G06T5/00 ,  G06T1/00 ,  G06T7/90 ,  G06T9/00 ,  G06T1/20 ,  G06N3/08 ,  G06N3/04

Abstract: Systems and techniques for estimating illumination from a single image are provided. An example system may include a neural network. The neural network may include an encoder that is configured to encode an input image into an intermediate representation. The neural network may also include an intensity decoder that is configured to decode the intermediate representation into an output light intensity map. An example intensity decoder is generated by a multi-phase training process that includes a first phase to train a light mask decoder using a set of low dynamic range images and a second phase to adjust parameters of the light mask decoder using a set of high dynamic range image to generate the intensity decoder.

公开(公告)号：US10546212B2

公开(公告)日：2020-01-28

申请号：US16148166

申请日：2018-10-01

Applicant: Adobe Inc.

Inventor： Nathan Carr ,  Kalyan Sunkavalli ,  Michal Lukac ,  Elya Shechtman

IPC: G06K9/62

Abstract: The present disclosure is directed toward systems and methods for image patch matching. In particular, the systems and methods described herein sample image patches to identify those image patches that match a target image patch. The systems and methods described herein probabilistically accept image patch proposals as potential matches based on an oracle. The oracle is computationally inexpensive to evaluate but more approximate than similarity heuristics. The systems and methods use the oracle to quickly guide the search to areas of the search space more likely to have a match. Once areas are identified that likely include a match, the systems and methods use a more accurate similarity function to identify patch matches.

公开(公告)号：US10467777B2

公开(公告)日：2019-11-05

申请号：US15934629

申请日：2018-03-23

Applicant: Adobe Inc.

Inventor： Siying Liu ,  Kalyan Sunkavalli ,  Nathan A. Carr ,  Elya Shechtman

IPC: G06T11/00 ,  G06K9/52 ,  G06K9/62 ,  G06T7/49

Abstract: Texture modeling techniques for image data are described. In one or more implementations, texels in image data are discovered by one or more computing devices, each texel representing an element that repeats to form a texture pattern in the image data. Regularity of the texels in the image data is modeled by the one or more computing devices to define translations and at least one other transformation of texels in relation to each other.

公开(公告)号：US12198231B2

公开(公告)日：2025-01-14

申请号：US18341618

申请日：2023-06-26

Applicant: Adobe Inc.

Inventor： Milos Hasan ,  Liang Shi ,  Tamy Boubekeur ,  Kalyan Sunkavalli ,  Radomir Mech

IPC: G06T11/00 ,  G06N3/08 ,  G06N3/084 ,  G06T11/40 ,  G06T15/04

Abstract: The present disclosure relates to using end-to-end differentiable pipeline for optimizing parameters of a base procedural material to generate a procedural material corresponding to a target physical material. For example, the disclosed systems can receive a digital image of a target physical material. In response, the disclosed systems can retrieve a differentiable procedural material for use as a base procedural material in response. The disclosed systems can compare a digital image of the base procedural material with the digital image of the target physical material using a loss function, such as a style loss function that compares visual appearance. Based on the determined loss, the disclosed systems can modify the parameters of the base procedural material to determine procedural material parameters for the target physical material. The disclosed systems can generate a procedural material corresponding to the base procedural material using the determined procedural material parameters.

公开(公告)号：US20240013477A1

公开(公告)日：2024-01-11

申请号：US17861199

申请日：2022-07-09

Applicant: Adobe Inc.

Inventor： Zexiang Xu ,  Zhixin Shu ,  Sai Bi ,  Qiangeng Xu ,  Kalyan Sunkavalli ,  Julien Philip

IPC: G06T15/20 ,  G06T15/80 ,  G06T15/06

CPC classification number: G06T15/205 ,  G06T15/80 ,  G06T15/06 ,  G06T2207/10028

Abstract: A scene modeling system receives a plurality of input two-dimensional (2D) images corresponding to a plurality of views of an object and a request to display a three-dimensional (3D) scene that includes the object. The scene modeling system generates an output 2D image for a view of the 3D scene by applying a scene representation model to the input 2D images. The scene representation model includes a point cloud generation model configured to generate, based on the input 2D images, a neural point cloud representing the 3D scene. The scene representation model includes a neural point volume rendering model configured to determine, for each pixel of the output image and using the neural point cloud and a volume rendering process, a color value. The scene modeling system transmits, responsive to the request, the output 2D image. Each pixel of the output image includes the respective determined color value.

公开(公告)号：US20230360327A1

公开(公告)日：2023-11-09

申请号：US17661878

申请日：2022-05-03

Applicant: Adobe Inc.

Inventor： Sai Bi ,  Yang Liu ,  Zexiang Xu ,  Fujun Luan ,  Kalyan Sunkavalli

IPC: G06T17/20 ,  G06T13/20

CPC classification number: G06T17/205 ,  G06T13/20 ,  G06T2210/21

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate three-dimensional hybrid mesh-volumetric representations for digital objects. For instance, in one or more embodiments, the disclosed systems generate a mesh for a digital object from a plurality of digital images that portray the digital object using a multi-view stereo model. Additionally, the disclosed systems determine a set of sample points for a thin volume around the mesh. Using a neural network, the disclosed systems further generate a three-dimensional hybrid mesh-volumetric representation for the digital object utilizing the set of sample points for the thin volume and the mesh.

公开(公告)号：US11783184B2

公开(公告)日：2023-10-10

申请号：US17590995

申请日：2022-02-02

Applicant: Adobe Inc.

Inventor： Federico Perazzi ,  Zhihao Xia ,  Michael Gharbi ,  Kalyan Sunkavalli

IPC: G06N3/08 ,  G06T15/50 ,  G06T5/00 ,  G06N20/10

CPC classification number: G06N3/08 ,  G06N20/10 ,  G06T5/002 ,  G06T15/50

Abstract: Certain embodiments involve techniques for efficiently estimating denoising kernels for generating denoised images. For instance, a neural network receives a noisy reference image to denoise. The neural network uses a kernel dictionary of base kernels and generates a coefficient vector for each pixel in the reference image such that the coefficient vector includes a coefficient value for each base kernel in the kernel dictionary, where the base kernels are combined to generate a denoising kernel and each coefficient value indicates a contribution of a given base kernel to a denoising kernel. The neural network calculates the denoising kernel for a given pixel by applying the coefficient vector for that pixel to the kernel dictionary. The neural network applies each denoising kernel to the respective pixel to generate a denoised output image.

公开(公告)号：US20230298148A1

公开(公告)日：2023-09-21

申请号：US17655663

申请日：2022-03-21

Applicant: Adobe Inc.

Inventor： He Zhang ,  Jianming Zhang ,  Jose Ignacio Echevarria Vallespi ,  Kalyan Sunkavalli ,  Meredith Payne Stotzner ,  Yinglan Ma ,  Zhe Lin ,  Elya Shechtman ,  Frederick Mandia

IPC: G06T5/50 ,  G06T7/194 ,  G06T7/90 ,  G06T11/00

CPC classification number: G06T5/50 ,  G06T7/194 ,  G06T7/90 ,  G06T11/001 ,  G06T2207/20084 ,  G06T2207/20212 ,  G06T2200/24 ,  G06T2207/20092 ,  G06T2207/20016 ,  G06T2207/20081 ,  G06T2207/30168

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a dual-branched neural network architecture to harmonize composite images. For example, in one or more implementations, the transformer-based harmonization system uses a convolutional branch and a transformer branch to generate a harmonized composite image based on an input composite image and a corresponding segmentation mask. More particularly, the convolutional branch comprises a series of convolutional neural network layers followed by a style normalization layer to extract localized information from the input composite image. Further, the transformer branch comprises a series of transformer neural network layers to extract global information based on different resolutions of the input composite image. Utilizing a decoder, the transformer-based harmonization system combines the local information and the global information from the corresponding convolutional branch and transformer branch to generate a harmonized composite image.

公开(公告)号：US11688109B2

公开(公告)日：2023-06-27

申请号：US17513747

申请日：2021-10-28

Applicant: Adobe Inc.

Inventor： Milos Hasan ,  Liang Shi ,  Tamy Boubekeur ,  Kalyan Sunkavalli ,  Radomir Mech

IPC: G06T11/00 ,  G06T11/40 ,  G06T15/04 ,  G06N3/084

CPC classification number: G06T11/001 ,  G06N3/084 ,  G06T11/40 ,  G06T15/04

Abstract: The present disclosure relates to using end-to-end differentiable pipeline for optimizing parameters of a base procedural material to generate a procedural material corresponding to a target physical material. For example, the disclosed systems can receive a digital image of a target physical material. In response, the disclosed systems can retrieve a differentiable procedural material for use as a base procedural material in response. The disclosed systems can compare a digital image of the base procedural material with the digital image of the target physical material using a loss function, such as a style loss function that compares visual appearance. Based on the determined loss, the disclosed systems can modify the parameters of the base procedural material to determine procedural material parameters for the target physical material. The disclosed systems can generate a procedural material corresponding to the base procedural material using the determined procedural material parameters.