公开(公告)号：US11861762B2

公开(公告)日：2024-01-02

申请号：US17400474

申请日：2021-08-12

Applicant: Adobe Inc.

Inventor： Yuheng Li ,  Yijun Li ,  Jingwan Lu ,  Elya Shechtman ,  Krishna Kumar Singh

IPC: G06T11/00

CPC classification number: G06T11/00 ,  G06T2210/12

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate synthetized digital images using class-specific generators for objects of different classes. The disclosed system modifies a synthesized digital image by utilizing a plurality of class-specific generator neural networks to generate a plurality of synthesized objects according to object classes identified in the synthesized digital image. The disclosed system determines object classes in the synthesized digital image such as via a semantic label map corresponding to the synthesized digital image. The disclosed system selects class-specific generator neural networks corresponding to the classes of objects in the synthesized digital image. The disclosed system also generates a plurality of synthesized objects utilizing the class-specific generator neural networks based on contextual data associated with the identified objects. The disclosed system generates a modified synthesized digital image by replacing the identified objects in the synthesized digital images with the synthesized objects.

公开(公告)号：US20220392025A1

公开(公告)日：2022-12-08

申请号：US17338949

申请日：2021-06-04

Applicant: Adobe Inc.

Inventor： Ionut Mironica ,  Yijun Li

IPC: G06T5/00 ,  G06T7/11 ,  G06T3/00 ,  G06T5/50 ,  G06N3/04 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly restoring degraded digital images utilizing a deep learning framework for repairing local defects, correcting global imperfections, and/or enhancing depicted faces. In particular, the disclosed systems can utilize a defect detection neural network to generate a segmentation map indicating locations of local defects within a digital image. In addition, the disclosed systems can utilize an inpainting algorithm to determine pixels for inpainting the local defects to reduce their appearance. In some embodiments, the disclosed systems utilize a global correction neural network to determine and repair global imperfections. Further, the disclosed systems can enhance one or more faces depicted within a digital image utilizing a face enhancement neural network as well.

公开(公告)号：US10482639B2

公开(公告)日：2019-11-19

申请号：US15438147

申请日：2017-02-21

Applicant: Adobe Inc.

Inventor： Yijun Li ,  Chen Fang ,  Jimei Yang ,  Zhaowen Wang ,  Xin Lu

IPC: G06T11/60 ,  G06K9/62 ,  G06K9/46 ,  G06T11/00

Abstract: In some embodiments, techniques for synthesizing an image style based on a plurality of neural networks are described. A computer system selects a style image based on user input that identifies the style image. The computer system generates an image based on a generator neural network and a loss neural network. The generator neural network outputs the synthesized image based on a noise vector and the style image and is trained based on style features generated from the loss neural network. The loss neural network outputs the style features based on a training image. The training image and the style image have a same resolution. The style features are generated at different resolutions of the training image. The computer system provides the synthesized image to a user device in response to the user input.

公开(公告)号：US10445921B1

公开(公告)日：2019-10-15

申请号：US16007898

申请日：2018-06-13

Applicant: Adobe Inc.

Inventor： Yijun Li ,  Chen Fang ,  Jimei Yang ,  Zhaowen Wang ,  Xin Lu

IPC: G06T13/80 ,  G06T7/246

Abstract: Transferring motion between consecutive frames to a digital image is leveraged in a digital medium environment. A digital image and at least a portion of the digital video are exposed to a motion transfer model. The portion of the digital video includes at least a first digital video frame and a second digital video frame that is consecutive to the first digital video frame. Flow data between the first digital video frame and the second digital image frame is extracted, and the flow data is then processed to generate motion features representing motion between the first digital video frame and the second digital video frame. The digital image is processed to generate image features of the digital image. Motion of the digital video is then transferred to the digital image by combining the motion features with the image features to generate a next digital image frame for the digital image.

公开(公告)号：US20250117995A1

公开(公告)日：2025-04-10

申请号：US18481719

申请日：2023-10-05

Applicant: ADOBE INC.

Inventor： Yijun Li ,  Matheus Abrantes Gadelha ,  Krishna Kumar Singh ,  Soren Pirk

IPC: G06T11/60 ,  G06T5/00 ,  G06T7/50 ,  G06V10/26

Abstract: Methods, non-transitory computer readable media, apparatuses, and systems for image and depth map generation include receiving a prompt and encoding the prompt to obtain a guidance embedding. A machine learning model then generates an image and a depth map corresponding to the image based on the guidance embedding. The image and the depth map are each generated based on the guidance embedding.

公开(公告)号：US12272031B2

公开(公告)日：2025-04-08

申请号：US17725818

申请日：2022-04-21

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yuheng Li ,  Yijun Li ,  Jingwan Lu ,  Elya Shechtman

IPC: G06T5/70 ,  G06N3/045 ,  G06T5/77 ,  G06V10/74 ,  G06V10/82

Abstract: An image inpainting system is described that receives an input image that includes a masked region. From the input image, the image inpainting system generates a synthesized image that depicts an object in the masked region by selecting a first code that represents a known factor characterizing a visual appearance of the object and a second code that represents an unknown factor characterizing the visual appearance of the object apart from the known factor in latent space. The input image, the first code, and the second code are provided as input to a generative adversarial network that is trained to generate the synthesized image using contrastive losses. Different synthesized images are generated from the same input image using different combinations of first and second codes, and the synthesized images are output for display.

公开(公告)号：US20240233318A9

公开(公告)日：2024-07-11

申请号：US17971169

申请日：2022-10-21

Applicant: Adobe Inc.

Inventor： Yijun Li ,  Zhixin Shu ,  Zhen Zhu ,  Krishna Kumar Singh

IPC: G06V10/70 ,  G06N3/04 ,  G06T11/00 ,  G06T15/08

CPC classification number: G06V10/70 ,  G06N3/0454 ,  G06T11/001 ,  G06T15/08

Abstract: An image generation system implements a multi-branch GAN to generate images that each express visually similar content in a different modality. A generator portion of the multi-branch GAN includes multiple branches that are each tasked with generating one of the different modalities. A discriminator portion of the multi-branch GAN includes multiple fidelity discriminators, one for each of the generator branches, and a consistency discriminator, which constrains the outputs generated by the different generator branches to appear visually similar to one another. During training, outputs from each of the fidelity discriminators and the consistency discriminator are used to compute a non-saturating GAN loss. The non-saturating GAN loss is used to refine parameters of the multi-branch GAN during training until model convergence. The trained multi-branch GAN generates multiple images from a single input, where each of the multiple images depicts visually similar content expressed in a different modality.

公开(公告)号：US20240135572A1

公开(公告)日：2024-04-25

申请号：US18190636

申请日：2023-03-27

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yijun Li ,  Jingwan Lu ,  Duygu Ceylan Aksit ,  Yangtuanfeng Wang ,  Jimei Yang ,  Tobias Hinz

IPC: G06T7/70 ,  G06T7/40 ,  G06V10/44 ,  G06V10/771 ,  G06V10/80 ,  G06V10/82

CPC classification number: G06T7/70 ,  G06T7/40 ,  G06V10/44 ,  G06V10/771 ,  G06V10/806 ,  G06V10/82 ,  G06T2207/20081 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

公开(公告)号：US20240020810A1

公开(公告)日：2024-01-18

申请号：US18474588

申请日：2023-09-26

Applicant: Adobe Inc.

Inventor： Yijun Li ,  Ionut Mironica

IPC: G06T5/50 ,  G06T11/00 ,  G06T5/00 ,  G06T7/50 ,  G06N3/04

CPC classification number: G06T5/50 ,  G06T11/001 ,  G06T5/002 ,  G06T7/50 ,  G06N3/04 ,  G06T2207/20084 ,  G06T2200/24 ,  G06T2207/20221 ,  G06T2207/20081

Abstract: Techniques for generating style-transferred images are provided. In some embodiments, a content image, a style image, and a user input indicating one or more modifications that operate on style-transferred images are received. In some embodiments, an initial style-transferred image is generated using a machine learning model. In some examples, the initial style-transferred image comprises features associated with the style image applied to content included in the content image. In some embodiments, a modified style-transferred image is generated by modifying the initial style-transferred image based at least in part on the user input indicating the one or more modifications.

公开(公告)号：US11842468B2

公开(公告)日：2023-12-12

申请号：US17178681

申请日：2021-02-18

Applicant: Adobe Inc.

Inventor： Pei Wang ,  Yijun Li ,  Jingwan Lu ,  Krishna Kumar Singh

IPC: G06K9/00 ,  G06T5/50 ,  G06N3/04 ,  G06V10/75 ,  G06F18/22 ,  G06F18/24

CPC classification number: G06T5/50 ,  G06F18/22 ,  G06F18/24 ,  G06N3/04 ,  G06V10/751 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06V10/759

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize image-guided model inversion of an image classifier with a discriminator. The disclosed systems utilize a neural network image classifier to encode features of an initial image and a target image. The disclosed system also reduces a feature distance between the features of the initial image and the features of the target image at a plurality of layers of the neural network image classifier by utilizing a feature distance regularizer. Additionally, the disclosed system reduces a patch difference between image patches of the initial image and image patches of the target image by utilizing a patch-based discriminator with a patch consistency regularizer. The disclosed system then generates a synthesized digital image based on the constrained feature set and constrained image patches of the initial image.