公开(公告)号：US20240161462A1

公开(公告)日：2024-05-16

申请号：US18053556

申请日：2022-11-08

Applicant: ADOBE INC.

Inventor： Yosef Gandelsman ,  Taesung Park ,  Richard Zhang ,  Elya Shechtman

IPC: G06V10/774 ,  G06T5/00 ,  G06T11/00 ,  G06V10/776 ,  G06V10/82 ,  G06V10/94

CPC classification number: G06V10/774 ,  G06T5/002 ,  G06T11/00 ,  G06V10/776 ,  G06V10/82 ,  G06V10/945 ,  G06T2200/24 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure include obtaining an image and a prompt for editing the image. A diffusion model is tuned based on the image to generate different versions of the image. The prompt is then encoded to obtain a guidance vector, and the diffusion model generates a modified image based on the image and the encoded text prompt.

公开(公告)号：US20240161327A1

公开(公告)日：2024-05-16

申请号：US18052658

申请日：2022-11-04

Applicant: ADOBE INC.

Inventor： Yinbo Chen ,  Michaël Gharbi ,  Oliver Wang ,  Richard Zhang ,  Elya Shechtman

IPC: G06T7/70 ,  G06T3/40 ,  G06T5/00 ,  G06T7/10

CPC classification number: G06T7/70 ,  G06T3/40 ,  G06T5/003 ,  G06T7/10 ,  G06T2207/20084 ,  G06T2207/20132 ,  G06T2207/20212

Abstract: Aspects of the methods, apparatus, non-transitory computer readable medium, and systems include obtaining a noise map and a global image code encoded from an original image and representing semantic content of the original image; generating a plurality of image patches based on the noise map and the global image code using a diffusion model; and combining the plurality of image patches to produce an output image including the semantic content.

公开(公告)号：US11930303B2

公开(公告)日：2024-03-12

申请号：US17526998

申请日：2021-11-15

Applicant: Adobe Inc.

Inventor： Pulkit Gera ,  Oliver Wang ,  Kalyan Krishna Sunkavalli ,  Elya Shechtman ,  Chetan Nanda

IPC: H04N9/31 ,  H04N9/73

CPC classification number: H04N9/3182 ,  G06T5/92 ,  H04N9/73 ,  G06T2207/20081

Abstract: Systems and techniques for automatic digital parameter adjustment are described that leverage insights learned from an image set to automatically predict parameter values for an input item of digital visual content. To do so, the automatic digital parameter adjustment techniques described herein captures visual and contextual features of digital visual content to determine balanced visual output in a range of visual scenes and settings. The visual and contextual features of digital visual content are used to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. The parameter adjustment model exploits these feature interactions to determine visually pleasing parameter values for an input item of digital visual content. The predicted parameter values are output, allowing further adjustment to the parameter values.

公开(公告)号：US11907839B2

公开(公告)日：2024-02-20

申请号：US17468511

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Ratheesh Kalarot ,  Kevin Wampler ,  Jingwan Lu ,  Jakub Fiser ,  Elya Shechtman ,  Aliakbar Darabi ,  Alexandru Vasile Costin

IPC: G06N3/08 ,  G06F3/04845 ,  G06T11/60 ,  G06T3/40 ,  G06T3/00 ,  G06F3/04847 ,  G06N20/20 ,  G06T5/00 ,  G06T5/20 ,  G06T11/00 ,  G06F18/40 ,  G06F18/211 ,  G06F18/214 ,  G06F18/21 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/0006 ,  G06T3/0093 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/005 ,  G06T5/20 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: Systems and methods combine an input image with an edited image generated using a generator neural network to preserve detail from the original image. A computing system provides an input image to a machine learning model to generate a latent space representation of the input image. The system provides the latent space representation to a generator neural network to generate a generated image. The system generates multiple scale representations of the input image, as well as multiple scale representations of the generated image. The system generates a first combined image based on first scale representations of the images and a first value. The system generates a second combined image based on second scale representations of the images and a second value. The system blends the first combined image with the second combined image to generate an output image.

公开(公告)号：US20240037922A1

公开(公告)日：2024-02-01

申请号：US17815451

申请日：2022-07-27

Applicant: Adobe Inc.

Inventor： Yijun Li ,  Nicholas Kolkin ,  Jingwan Lu ,  Elya Shechtman

IPC: G06V10/82 ,  G06V10/77 ,  G06V10/46

CPC classification number: G06V10/82 ,  G06V10/7715 ,  G06V10/469

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for adapting generative neural networks to target domains utilizing an image translation neural network. In particular, in one or more embodiments, the disclosed systems utilize an image translation neural network to translate target results to a source domain for input in target neural network adaptation. For instance, in some embodiments, the disclosed systems compare a translated target result with a source result from a pretrained source generative neural network to adjust parameters of a target generative neural network to produce results corresponding in features to source results and corresponding in style to the target domain.

公开(公告)号：US11875221B2

公开(公告)日：2024-01-16

申请号：US17468476

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Wei-An Lin ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Jun-Yan Zhu ,  Niloy Mitra ,  Ratheesh Kalarot ,  Richard Zhang ,  Shabnam Ghadar ,  Zhixin Shu

IPC: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06T11/60 ,  G06T3/40 ,  G06N20/20 ,  G06T5/00 ,  G06T5/20 ,  G06T3/00 ,  G06T11/00 ,  G06F18/40 ,  G06F18/211 ,  G06F18/214 ,  G06F18/21 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/0006 ,  G06T3/0093 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/005 ,  G06T5/20 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.

公开(公告)号：US20230360180A1

公开(公告)日：2023-11-09

申请号：US17661985

申请日：2022-05-04

Applicant: Adobe Inc.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jingwan Lu ,  Scott Cohen ,  Elya Shechtman ,  Connelly Barnes ,  Jianming Zhang ,  Ning Xu ,  Sohrab Amirghodsi

IPC: G06V10/40 ,  G06T3/40 ,  G06T5/00

CPC classification number: G06T5/005 ,  G06T3/4046 ,  G06V10/40 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing a cascaded modulation inpainting neural network. For example, the disclosed systems utilize a cascaded modulation inpainting neural network that includes cascaded modulation decoder layers. For example, in one or more decoder layers, the disclosed systems start with global code modulation that captures the global-range image structures followed by an additional modulation that refines the global predictions. Accordingly, in one or more implementations, the image inpainting system provides a mechanism to correct distorted local details. Furthermore, in one or more implementations, the image inpainting system leverages fast Fourier convolutions block within different resolution layers of the encoder architecture to expand the receptive field of the encoder and to allow the network encoder to better capture global structure.

公开(公告)号：US20230342884A1

公开(公告)日：2023-10-26

申请号：US17725818

申请日：2022-04-21

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yuheng Li ,  Yijun Li ,  Jingwan Lu ,  Elya Shechtman

IPC: G06T5/00 ,  G06V10/82 ,  G06V10/74 ,  G06N3/04

CPC classification number: G06T5/002 ,  G06V10/82 ,  G06V10/761 ,  G06N3/0454 ,  G06T2207/20081

Abstract: An image inpainting system is described that receives an input image that includes a masked region. From the input image, the image inpainting system generates a synthesized image that depicts an object in the masked region by selecting a first code that represents a known factor characterizing a visual appearance of the object and a second code that represents an unknown factor characterizing the visual appearance of the object apart from the known factor in latent space. The input image, the first code, and the second code are provided as input to a generative adversarial network that is trained to generate the synthesized image using contrastive losses. Different synthesized images are generated from the same input image using different combinations of first and second codes, and the synthesized images are output for display.

公开(公告)号：US11776188B2

公开(公告)日：2023-10-03

申请号：US17887685

申请日：2022-08-15

Applicant: Adobe Inc.

Inventor： Dingzeyu Li ,  Yang Zhou ,  Jose Ignacio Echevarria Vallespi ,  Elya Shechtman

IPC: G06T13/20 ,  G06T17/20 ,  G06T13/40

CPC classification number: G06T13/205 ,  G06T13/40 ,  G06T17/20

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for generating an animation of a talking head from an input audio signal of speech and a representation (such as a static image) of a head to animate. Generally, a neural network can learn to predict a set of 3D facial landmarks that can be used to drive the animation. In some embodiments, the neural network can learn to detect different speaking styles in the input speech and account for the different speaking styles when predicting the 3D facial landmarks. Generally, template 3D facial landmarks can be identified or extracted from the input image or other representation of the head, and the template 3D facial landmarks can be used with successive windows of audio from the input speech to predict 3D facial landmarks and generate a corresponding animation with plausible 3D effects.

公开(公告)号：US20230298148A1

公开(公告)日：2023-09-21

申请号：US17655663

申请日：2022-03-21

Applicant: Adobe Inc.

Inventor： He Zhang ,  Jianming Zhang ,  Jose Ignacio Echevarria Vallespi ,  Kalyan Sunkavalli ,  Meredith Payne Stotzner ,  Yinglan Ma ,  Zhe Lin ,  Elya Shechtman ,  Frederick Mandia

IPC: G06T5/50 ,  G06T7/194 ,  G06T7/90 ,  G06T11/00

CPC classification number: G06T5/50 ,  G06T7/194 ,  G06T7/90 ,  G06T11/001 ,  G06T2207/20084 ,  G06T2207/20212 ,  G06T2200/24 ,  G06T2207/20092 ,  G06T2207/20016 ,  G06T2207/20081 ,  G06T2207/30168

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a dual-branched neural network architecture to harmonize composite images. For example, in one or more implementations, the transformer-based harmonization system uses a convolutional branch and a transformer branch to generate a harmonized composite image based on an input composite image and a corresponding segmentation mask. More particularly, the convolutional branch comprises a series of convolutional neural network layers followed by a style normalization layer to extract localized information from the input composite image. Further, the transformer branch comprises a series of transformer neural network layers to extract global information based on different resolutions of the input composite image. Utilizing a decoder, the transformer-based harmonization system combines the local information and the global information from the corresponding convolutional branch and transformer branch to generate a harmonized composite image.