公开(公告)号：US20230316606A1

公开(公告)日：2023-10-05

申请号：US17655739

申请日：2022-03-21

Applicant: Adobe Inc.

Inventor： Hui Qu ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Ratheesh Kalarot ,  Richard Zhang ,  Saeid Motiian ,  Shabnam Ghadar ,  Wei-An Lin

IPC: G06N3/04 ,  G06T11/60

CPC classification number: G06T11/60 ,  G06N3/0454

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for latent-based editing of digital images using a generative neural network. In particular, in one or more embodiments, the disclosed systems perform latent-based editing of a digital image by mapping a feature tensor and a set of style vectors for the digital image into a joint feature style space. In one or more implementations, the disclosed systems apply a joint feature style perturbation and/or modification vectors within the joint feature style space to determine modified style vectors and a modified feature tensor. Moreover, in one or more embodiments the disclosed systems generate a modified digital image utilizing a generative neural network from the modified style vectors and the modified feature tensor.

公开(公告)号：US20230316474A1

公开(公告)日：2023-10-05

申请号：US17657691

申请日：2022-04-01

Applicant: Adobe Inc.

Inventor： Hui Qu ,  Jingwan Lu ,  Saeid Motiian ,  Shabnam Ghadar ,  Wei-An Lin ,  Elya Shechtman

IPC: G06T5/50 ,  G06T7/11 ,  G06N3/04

CPC classification number: G06T5/50 ,  G06T7/11 ,  G06N3/0454 ,  G06T2207/20172 ,  G06T2207/20084

Abstract: Methods, systems, and non-transitory computer readable media are disclosed for intelligently enhancing details in edited images. The disclosed system iteratively updates residual detail latent code for segments in edited images where detail has been lost through the editing process. More particularly, the disclosed system enhances an edited segment in an edited image based on details in a detailed segment of an image. Additionally, the disclosed system may utilize a detail neural network encoder to project the detailed segment and a corresponding segment of the edited image into a residual detail latent code. In some embodiments, the disclosed system generates a refined edited image based on the residual detail latent code and a latent vector of the edited image.

公开(公告)号：US11756210B2

公开(公告)日：2023-09-12

申请号：US16817100

申请日：2020-03-12

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  Matthew Fisher ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman ,  Wenqi Xian

IPC: G06V20/40 ,  G06T7/246 ,  G06N20/00 ,  G06V10/44 ,  G06V10/56

CPC classification number: G06T7/246 ,  G06N20/00 ,  G06V10/44 ,  G06V10/56 ,  G06V20/41 ,  G06V20/46

Abstract: Certain aspects involve video inpainting in which content is propagated from a user-provided reference frame to other video frames depicting a scene. For example, a computing system accesses a set of video frames with annotations identifying a target region to be modified. The computing system determines a motion of the target region's boundary across the set of video frames, and also interpolates pixel motion within the target region across the set of video frames. The computing system also inserts, responsive to user input, a reference frame into the set of video frames. The reference frame can include reference color data from a user-specified modification to the target region. The computing system can use the reference color data and the interpolated motion to update color data in the target region across set of video frames.

公开(公告)号：US20230274535A1

公开(公告)日：2023-08-31

申请号：US17680906

申请日：2022-02-25

Applicant: ADOBE INC.

Inventor： Yijun Li ,  Utkarsh Ojha ,  Richard Zhang ,  Jingwan Lu ,  Elya Shechtman ,  Alexei A. Efros

IPC: G06V10/774 ,  G06F3/04842

CPC classification number: G06V10/7747 ,  G06F3/04842

Abstract: An image generation system enables user input during the process of training a generative model to influence the model's ability to generate new images with desired visual features. A source generative model for a source domain is fine-tuned using training images in a target domain to provide an adapted generative model for the target domain. Interpretable factors are determined for the source generative model and the adapted generative model. A user interface is provided that enables a user to select one or more interpretable factors. The user-selected interpretable factor(s) are used to generate a user-adapted generative model, for instance, by using a loss function based on the user-selected interpretable factor(s). The user-adapted generative model can be used to create new images in the target domain.

公开(公告)号：US20230051749A1

公开(公告)日：2023-02-16

申请号：US17400474

申请日：2021-08-12

Applicant: Adobe Inc.

Inventor： Yuheng Li ,  Yijun Li ,  Jingwan Lu ,  Elya Shechtman ,  Krishna Kumar Singh

IPC: G06T11/00

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate synthetized digital images using class-specific generators for objects of different classes. The disclosed system modifies a synthesized digital image by utilizing a plurality of class-specific generator neural networks to generate a plurality of synthesized objects according to object classes identified in the synthesized digital image. The disclosed system determines object classes in the synthesized digital image such as via a semantic label map corresponding to the synthesized digital image. The disclosed system selects class-specific generator neural networks corresponding to the classes of objects in the synthesized digital image. The disclosed system also generates a plurality of synthesized objects utilizing the class-specific generator neural networks based on contextual data associated with the identified objects. The disclosed system generates a modified synthesized digital image by replacing the identified objects in the synthesized digital images with the synthesized objects.

公开(公告)号：US20220398712A1

公开(公告)日：2022-12-15

申请号：US17820649

申请日：2022-08-18

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Aliakbar Darabi ,  Elya Shechtman

IPC: G06T7/00 ,  G06T3/40 ,  G06T9/00 ,  G06V10/75

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating modified digital images by utilizing a patch match algorithm to generate nearest neighbor fields for a second digital image based on a nearest neighbor field associated with a first digital image. For example, the disclosed systems can identify a nearest neighbor field associated with a first digital image of a first resolution. Based on the nearest neighbor field of the first digital image, the disclosed systems can utilize a patch match algorithm to generate a nearest neighbor field for a second digital image of a second resolution larger than the first resolution. The disclosed systems can further generate a modified digital image by filling a target region of the second digital image utilizing the generated nearest neighbor field.

公开(公告)号：US20220292650A1

公开(公告)日：2022-09-15

申请号：US17202019

申请日：2021-03-15

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Lingzhi Zhang ,  Zhe Lin ,  Connelly Barnes ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/50 ,  G06T7/11 ,  G06T3/40 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly generating modified digital images utilizing a guided inpainting approach that implements a patch match model informed by a deep visual guide. In particular, the disclosed systems can utilize a visual guide algorithm to automatically generate guidance maps to help identify replacement pixels for inpainting regions of digital images utilizing a patch match model. For example, the disclosed systems can generate guidance maps in the form of structure maps, depth maps, or segmentation maps that respectively indicate the structure, depth, or segmentation of different portions of digital images. Additionally, the disclosed systems can implement a patch match model to identify replacement pixels for filling regions of digital images according to the structure, depth, and/or segmentation of the digital images.

公开(公告)号：US11417041B2

公开(公告)日：2022-08-16

申请号：US16788551

申请日：2020-02-12

Applicant: ADOBE INC.

Inventor： Dingzeyu Li ,  Yang Zhou ,  Jose Ignacio Echevarria Vallespi ,  Elya Shechtman

IPC: G06T13/20 ,  G06T13/40 ,  G06T17/20

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for generating an animation of a talking head from an input audio signal of speech and a representation (such as a static image) of a head to animate. Generally, a neural network can learn to predict a set of 3D facial landmarks that can be used to drive the animation. In some embodiments, the neural network can learn to detect different speaking styles in the input speech and account for the different speaking styles when predicting the 3D facial landmarks. Generally, template 3D facial landmarks can be identified or extracted from the input image or other representation of the head, and the template 3D facial landmarks can be used with successive windows of audio from the input speech to predict 3D facial landmarks and generate a corresponding animation with plausible 3D effects.

公开(公告)号：US20220172331A1

公开(公告)日：2022-06-02

申请号：US17651435

申请日：2022-02-17

Applicant: Adobe Inc.

Inventor： Connelly Barnes ,  Sohrab Amirghodsi ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/33 ,  G06T5/50

Abstract: Techniques are disclosed for filling or otherwise replacing a target region of a primary image with a corresponding region of an auxiliary image. The filling or replacing can be done with an overlay (no subtractive process need be run on the primary image). Because the primary and auxiliary images may not be aligned, both geometric and photometric transformations are applied to the primary and/or auxiliary images. For instance, a geometric transformation of the auxiliary image is performed, to better align features of the auxiliary image with corresponding features of the primary image. Also, a photometric transformation of the auxiliary image is performed, to better match color of one or more pixels of the auxiliary image with color of corresponding one or more pixels of the primary image. The corresponding region of the transformed auxiliary image is then copied and overlaid on the target region of the primary image.

公开(公告)号：US20220122306A1

公开(公告)日：2022-04-21

申请号：US17468487

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Wei-An Lin ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Jun-Yan Zhu ,  Niloy Mitra ,  Ratheesh Kalarot ,  Richard Zhang ,  Shabnam Ghadar ,  Zhixin Shu

IPC: G06T11/60 ,  G06F3/0484 ,  G06N3/08 ,  G06N3/04

Abstract: Systems and methods dynamically adjust an available range for editing an attribute in an image. An image editing system computes a metric for an attribute in an input image as a function of a latent space representation of the input image and a filtering vector for editing the input image. The image editing system compares the metric to a threshold. If the metric exceeds the threshold, then the image editing system selects a first range for editing the attribute in the input image. If the metric does not exceed the threshold, a second range is selected. The image editing system causes display of a user interface for editing the input image comprising an interface element for editing the attribute within the selected range.