公开(公告)号：US20220122221A1

公开(公告)日：2022-04-21

申请号：US17384357

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Cameron Smith ,  Ratheesh Kalarot ,  Wei-An Lin ,  Richard Zhang ,  Niloy Mitra ,  Elya Shechtman ,  Shabnam Ghadar ,  Zhixin Shu ,  Yannick Hold-Geoffrey ,  Nathan Carr ,  Jingwan Lu ,  Oliver Wang ,  Jun-Yan Zhu

IPC: G06T3/40 ,  G06F3/0484 ,  G06N3/08 ,  G06N3/04

Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.

公开(公告)号：US20220121931A1

公开(公告)日：2022-04-21

申请号：US17384371

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Ratheesh Kalarot ,  Wei-An Lin ,  Cameron Smith ,  Zhixin Shu ,  Baldo Faieta ,  Shabnam Ghadar ,  Jingwan Lu ,  Aliakbar Darabi ,  Jun-Yan Zhu ,  Niloy Mitra ,  Richard Zhang ,  Elya Shechtman

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62

Abstract: Systems and methods train and apply a specialized encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The specialized encoder neural network includes an input layer, a feature extraction layer, and a bottleneck layer positioned after the feature extraction layer. The projection process includes providing an input image to the encoder and producing, by the encoder, a latent space representation of the input image. Producing the latent space representation includes extracting a feature vector from the feature extraction layer, providing the feature vector to the bottleneck layer as input, and producing the latent space representation as output. The latent space representation produced by the encoder is provided as input to the GAN, which generates an output image based upon the latent space representation. The encoder is trained using specialized loss functions including a segmentation loss and a mean latent loss.

公开(公告)号：US12211138B2

公开(公告)日：2025-01-28

申请号：US18065456

申请日：2022-12-13

Applicant: Adobe Inc.

Inventor： Zhengfei Kuang ,  Fujun Luan ,  Sai Bi ,  Zhixin Shu ,  Kalyan K. Sunkavalli

IPC: G06T15/08 ,  G06T15/06 ,  G06T15/50 ,  G06T15/80 ,  G06T19/20

Abstract: Embodiments of the present disclosure provide systems, methods, and computer storage media for generating editable synthesized views of scenes by inputting image rays into neural networks using neural basis decomposition. In embodiments, a set of input images of a scene depicting at least one object are collected and used to generate a plurality of rays of the scene. The rays each correspond to three-dimensional coordinates and viewing angles taken from the images. A volume density of the scene is determined by inputting the three-dimensional coordinates from the neural radiance fields into a first neural network to generate a 3D geometric representation of the object. An appearance decomposition is produced by inputting the three-dimensional coordinates and the viewing angles of the rays into a second neural network.

公开(公告)号：US20240404188A1

公开(公告)日：2024-12-05

申请号：US18205279

申请日：2023-06-02

Applicant: Adobe Inc.

Inventor： He Zhang ,  Zijun Wei ,  Zhixin Shu ,  Yiqun Mei ,  Yilin Wang ,  Xuaner Zhang ,  Shi Yan ,  Jianming Zhang

IPC: G06T15/50 ,  G06T7/11 ,  G06T15/04 ,  G06T15/60 ,  G06T15/80

Abstract: In accordance with the described techniques, a portrait relighting system receives user input defining one or more markings drawn on a portrait image. Using one or more machine learning models, the portrait relighting system generates an albedo representation of the portrait image by removing lighting effects from the portrait image. Further, the portrait relighting system generates a shading map of the portrait image using the one or more machine learning models by designating the one or more markings as a lighting condition, and applying the lighting condition to a geometric representation of the portrait image. The one or more machine learning models are further employed to generate a relit portrait image based on the albedo representation and the shading map.

公开(公告)号：US11934958B2

公开(公告)日：2024-03-19

申请号：US17147912

申请日：2021-01-13

Applicant: Adobe Inc.

Inventor： Zhixin Shu ,  Zhe Lin ,  Yuchen Liu ,  Yijun Li

IPC: G06N3/08 ,  G06N3/082 ,  G06N3/084 ,  G06T5/00 ,  G06T5/40 ,  G06T7/194

CPC classification number: G06N3/082 ,  G06N3/084 ,  G06T5/002 ,  G06T5/004 ,  G06T5/40 ,  G06T7/194

Abstract: This disclosure describes one or more embodiments of systems, non-transitory computer-readable media, and methods that utilize channel pruning and knowledge distillation to generate a compact noise-to-image GAN. For example, the disclosed systems prune less informative channels via outgoing channel weights of the GAN. In some implementations, the disclosed systems further utilize content-aware pruning by utilizing a differentiable loss between an image generated by the GAN and a modified version of the image to identify sensitive channels within the GAN during channel pruning. In some embodiments, the disclosed systems utilize knowledge distillation to learn parameters for the pruned GAN to mimic a full-size GAN. In certain implementations, the disclosed systems utilize content-aware knowledge distillation by applying content masks on images generated by both the pruned GAN and its full-size counterpart to obtain knowledge distillation losses between the images for use in learning the parameters for the pruned GAN.

公开(公告)号：US11875221B2

公开(公告)日：2024-01-16

申请号：US17468476

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Wei-An Lin ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Jun-Yan Zhu ,  Niloy Mitra ,  Ratheesh Kalarot ,  Richard Zhang ,  Shabnam Ghadar ,  Zhixin Shu

IPC: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06T11/60 ,  G06T3/40 ,  G06N20/20 ,  G06T5/00 ,  G06T5/20 ,  G06T3/00 ,  G06T11/00 ,  G06F18/40 ,  G06F18/211 ,  G06F18/214 ,  G06F18/21 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/0006 ,  G06T3/0093 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/005 ,  G06T5/20 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.

公开(公告)号：US20240013477A1

公开(公告)日：2024-01-11

申请号：US17861199

申请日：2022-07-09

Applicant: Adobe Inc.

Inventor： Zexiang Xu ,  Zhixin Shu ,  Sai Bi ,  Qiangeng Xu ,  Kalyan Sunkavalli ,  Julien Philip

IPC: G06T15/20 ,  G06T15/80 ,  G06T15/06

CPC classification number: G06T15/205 ,  G06T15/80 ,  G06T15/06 ,  G06T2207/10028

Abstract: A scene modeling system receives a plurality of input two-dimensional (2D) images corresponding to a plurality of views of an object and a request to display a three-dimensional (3D) scene that includes the object. The scene modeling system generates an output 2D image for a view of the 3D scene by applying a scene representation model to the input 2D images. The scene representation model includes a point cloud generation model configured to generate, based on the input 2D images, a neural point cloud representing the 3D scene. The scene representation model includes a neural point volume rendering model configured to determine, for each pixel of the output image and using the neural point cloud and a volume rendering process, a color value. The scene modeling system transmits, responsive to the request, the output 2D image. Each pixel of the output image includes the respective determined color value.

公开(公告)号：US20230360299A1

公开(公告)日：2023-11-09

申请号：US18224916

申请日：2023-07-21

Applicant: Adobe Inc.

Inventor： Yang Yang ,  Zhixin Shu ,  Shabnam Ghadar ,  Jingwan Lu ,  Jakub Fiser ,  Elya Schechtman ,  Cameron Y. Smith ,  Baldo Antonio Faieta ,  Alex Charles Filipkowski

IPC: G06T11/60 ,  G06F21/62 ,  G06F16/56 ,  G06F16/532

CPC classification number: G06T11/60 ,  G06F21/6254 ,  G06F16/56 ,  G06F16/532 ,  G06T2200/24

Abstract: Face anonymization techniques are described that overcome conventional challenges to generate an anonymized face. In one example, a digital object editing system is configured to generate an anonymized face based on a target face and a reference face. As part of this, the digital object editing system employs an encoder as part of machine learning to extract a target encoding of the target face image and a reference encoding of the reference face. The digital object editing system then generates a mixed encoding from the target and reference encodings. The mixed encoding is employed by a machine-learning model of the digital object editing system to generate a mixed face. An object replacement module is used by the digital object editing system to replace the target face in the target digital image with the mixed face.

公开(公告)号：US20220148243A1

公开(公告)日：2022-05-12

申请号：US17094093

申请日：2020-11-10

Applicant: Adobe Inc.

Inventor： Yang Yang ,  Zhixin Shu ,  Shabnam Ghadar ,  Jingwan Lu ,  Jakub Fiser ,  Elya Schechtman ,  Cameron Y. Smith ,  Baldo Antonio Faieta ,  Alex Charles Filipkowski

IPC: G06T11/60 ,  G06T9/00 ,  G06F3/0484 ,  G06F16/532 ,  G06F21/62 ,  G06F16/56 ,  G06N20/00

Abstract: Face anonymization techniques are described that overcome conventional challenges to generate an anonymized face. In one example, a digital object editing system is configured to generate an anonymized face based on a target face and a reference face. As part of this, the digital object editing system employs an encoder as part of machine learning to extract a target encoding of the target face image and a reference encoding of the reference face. The digital object editing system then generates a mixed encoding from the target and reference encodings. The mixed encoding is employed by a machine-learning model of the digital object editing system to generate a mixed face. An object replacement module is used by the digital object editing system to replace the target face in the target digital image with the mixed face.

公开(公告)号：US20240404181A1

公开(公告)日：2024-12-05

申请号：US18799247

申请日：2024-08-09

Applicant: Adobe Inc.

Inventor： Zexiang Xu ,  Zhixin Shu ,  Sai Bi ,  Qiangeng Xu ,  Kalyan Sunkavalli ,  Julien Philip

IPC: G06T15/20 ,  G06T15/06 ,  G06T15/80

Abstract: A scene modeling system receives a plurality of input two-dimensional (2D) images corresponding to a plurality of views of an object and a request to display a three-dimensional (3D) scene that includes the object. The scene modeling system generates an output 2D image for a view of the 3D scene by applying a scene representation model to the input 2D images. The scene representation model includes a point cloud generation model configured to generate, based on the input 2D images, a neural point cloud representing the 3D scene. The scene representation model includes a neural point volume rendering model configured to determine, for each pixel of the output image and using the neural point cloud and a volume rendering process, a color value. The scene modeling system transmits, responsive to the request, the output 2D image. Each pixel of the output image includes the respective determined color value.