公开(公告)号：US20220122221A1

公开(公告)日：2022-04-21

申请号：US17384357

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Cameron Smith ,  Ratheesh Kalarot ,  Wei-An Lin ,  Richard Zhang ,  Niloy Mitra ,  Elya Shechtman ,  Shabnam Ghadar ,  Zhixin Shu ,  Yannick Hold-Geoffrey ,  Nathan Carr ,  Jingwan Lu ,  Oliver Wang ,  Jun-Yan Zhu

IPC: G06T3/40 ,  G06F3/0484 ,  G06N3/08 ,  G06N3/04

Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.

公开(公告)号：US20220121931A1

公开(公告)日：2022-04-21

申请号：US17384371

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Ratheesh Kalarot ,  Wei-An Lin ,  Cameron Smith ,  Zhixin Shu ,  Baldo Faieta ,  Shabnam Ghadar ,  Jingwan Lu ,  Aliakbar Darabi ,  Jun-Yan Zhu ,  Niloy Mitra ,  Richard Zhang ,  Elya Shechtman

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62

Abstract: Systems and methods train and apply a specialized encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The specialized encoder neural network includes an input layer, a feature extraction layer, and a bottleneck layer positioned after the feature extraction layer. The projection process includes providing an input image to the encoder and producing, by the encoder, a latent space representation of the input image. Producing the latent space representation includes extracting a feature vector from the feature extraction layer, providing the feature vector to the bottleneck layer as input, and producing the latent space representation as output. The latent space representation produced by the encoder is provided as input to the GAN, which generates an output image based upon the latent space representation. The encoder is trained using specialized loss functions including a segmentation loss and a mean latent loss.

公开(公告)号：US10380317B2

公开(公告)日：2019-08-13

申请号：US15063183

申请日：2016-03-07

Applicant: Adobe Inc.

Inventor： Duygu Ceylan ,  Byungmoon Kim ,  Aron Monszpart ,  Vladimir Kim ,  Niloy Mitra

IPC: G06F17/50

Abstract: Methods and systems for generating digital models from objects. In particular, one or more embodiments determine a plurality of correspondences for first and second components of an object. One or more embodiments estimate a joint connecting the first and second components based on the correspondences. One or more embodiments jointly determine a global transformation and one or more joint parameters that map the plurality of components of the object from the first digital scan to the second digital scan. One or more embodiments also updating the correspondences based on the determined global transformation and parameter(s). One or more embodiments re-estimate the joint based on the updated correspondences. One or more embodiments select a candidate joint with a lowest error estimate from a plurality of candidate joints according to determined global transformations and joint parameter(s) for the candidate joints.

公开(公告)号：US11640684B2

公开(公告)日：2023-05-02

申请号：US16934858

申请日：2020-07-21

Applicant: ADOBE INC. ,  KING ABDULLAH UNIVERSITY OF SCIENCE AND TECHNOLOGY

Inventor： Rameen Abdal ,  Peter Wonka ,  Niloy Mitra ,  Peihao Zhu

IPC: G06T11/60 ,  G06V40/16

Abstract: A method, apparatus, and non-transitory computer readable medium for image processing are described. Embodiments of the method, apparatus, and non-transitory computer readable medium include identifying an original image including a plurality of semantic attributes, wherein each of the semantic attributes represents a complex set of features of the original image; identifying a target attribute value that indicates a change to a target attribute of the semantic attributes; computing a modified feature vector based on the target attribute value, wherein the modified feature vector incorporates the change to the target attribute while holding at least one preserved attribute of the semantic attributes substantially unchanged; and generating a modified image based on the modified feature vector, wherein the modified image includes the change to the target attribute and retains the at least one preserved attribute from the original image.

公开(公告)号：US20220122305A1

公开(公告)日：2022-04-21

申请号：US17384273

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Cameron Smith ,  Ratheesh Kalarot ,  Wei-An Lin ,  Richard Zhang ,  Niloy Mitra ,  Elya Shechtman ,  Shabnam Ghadar ,  Zhixin Shu ,  Yannick Hold-Geoffrey ,  Nathan Carr ,  Jingwan Lu ,  Oliver Wang ,  Jun-Yan Zhu

IPC: G06T11/60 ,  G06T3/40

Abstract: An improved system architecture uses a pipeline including an encoder and a Generative Adversarial Network (GAN) including a generator neural network to generate edited images with improved speed, realism, and identity preservation. The encoder produces an initial latent space representation of an input image by encoding the input image. The generator neural network generates an initial output image by processing the initial latent space representation of the input image. The system generates an optimized latent space representation of the input image using a loss minimization technique that minimizes a loss between the input image and the initial output image. The loss is based on target perceptual features extracted from the input image and initial perceptual features extracted from the initial output image. The system outputs the optimized latent space representation of the input image for downstream use.

公开(公告)号：US20220122222A1

公开(公告)日：2022-04-21

申请号：US17384283

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Cameron Smith ,  Ratheesh Kalarot ,  Wei-An Lin ,  Richard Zhang ,  Niloy Mitra ,  Elya Shechtman ,  Shabnam Ghadar ,  Zhixin Shu ,  Yannick Hold-Geoffrey ,  Nathan Carr ,  Jingwan Lu ,  Oliver Wang ,  Jun-Yan Zhu

IPC: G06T3/40 ,  G06T11/60

Abstract: An improved system architecture uses a Generative Adversarial Network (GAN) including a specialized generator neural network to generate multiple resolution output images. The system produces a latent space representation of an input image. The system generates a first output image at a first resolution by providing the latent space representation of the input image as input to a generator neural network comprising an input layer, an output layer, and a plurality of intermediate layers and taking the first output image from an intermediate layer, of the plurality of intermediate layers of the generator neural network. The system generates a second output image at a second resolution different from the first resolution by providing the latent space representation of the input image as input to the generator neural network and taking the second output image from the output layer of the generator neural network.

公开(公告)号：US20220121932A1

公开(公告)日：2022-04-21

申请号：US17384378

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Ratheesh Kalarot ,  Wei-An Lin ,  Cameron Smith ,  Zhixin Shu ,  Baldo Faieta ,  Shabnam Ghadar ,  Jingwan Lu ,  Aliakbar Darabi ,  Jun-Yan Zhu ,  Niloy Mitra ,  Richard Zhang ,  Elya Shechtman

IPC: G06N3/08 ,  G06N3/04

Abstract: Systems and methods train an encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The encoder is trained by providing an input training image to the encoder and producing, by the encoder, a latent space representation of the input training image. The latent space representation is provided as input to the GAN to generate a generated training image. A latent code is sampled from a latent space associated with the GAN and the sampled latent code is provided as input to the GAN. The GAN generates a synthetic training image based on the sampled latent code. The sampled latent code is provided as input to the encoder to produce a synthetic training code. The encoder is updated by minimizing a loss between the generated training image and the input training image, and the synthetic training code and the sampled latent code.

公开(公告)号：US20200175759A1

公开(公告)日：2020-06-04

申请号：US16205132

申请日：2018-11-29

Applicant: Adobe Inc.

Inventor： Bryan Russell ,  Daniel Kaufman ,  Carlo Innamorati ,  Niloy Mitra

IPC: G06T19/00 ,  G06T15/20 ,  G06T17/00 ,  G06F3/01 ,  G06K9/62 ,  G06N20/00

Abstract: This application relates generally to augmenting images and videos with dynamic object compositing, and more specifically, to generating synthetic training data to train a machine learning model to automatically augment an image or video with a dynamic object. The synthetic training data may contain multiple data points from thousands of simulated dynamic object movements within a virtual environment. Based on the synthetic training data, the machine learning model may determine the movement of a new dynamic object within new virtual environment.