公开(公告)号：US20220148241A1

公开(公告)日：2022-05-12

申请号：US17091416

申请日：2020-11-06

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Alexei A. Efros ,  Elya Shechtman ,  Richard Zhang ,  Junyan Zhu

IPC: G06T11/60 ,  G06N3/04 ,  G06T7/10 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating modified digital images utilizing a novel swapping autoencoder that incorporates scene layout. In particular, the disclosed systems can receive a scene layout map that indicates or defines locations for displaying specific digital content within a digital image. In addition, the disclosed systems can utilize the scene layout map to guide combining portions of digital image latent code to generate a modified digital image with a particular textural appearance and a particular geometric structure defined by the scene layout map. Additionally, the disclosed systems can utilize a scene layout map that defines a portion of a digital image to modify by, for instance, adding new digital content to the digital image, and can generate a modified digital image depicting the new digital content.

公开(公告)号：US12271996B2

公开(公告)日：2025-04-08

申请号：US18166189

申请日：2023-02-08

Applicant: ADOBE INC.

Inventor： Sudeep Siddheshwar Katakol ,  Taesung Park ,  Aliakbar Darabi ,  Kevin Duarte ,  Ryan Joe Murdock

IPC: G06T15/50 ,  G06T11/60

Abstract: A method for training a GAN to transfer lighting from a reference image to a source image includes: receiving the source image and the reference image; generating a lighting vector from the reference image; applying features of the source image and the lighting vector to a generative network of the GAN to create a generated image; applying features of the reference image and the lighting vector to a discriminative network of the GAN to update weights of the discriminative network; and applying features of the generated image and the lighting vector to the discriminative network to update weights of the generative network.

公开(公告)号：US12192593B2

公开(公告)日：2025-01-07

申请号：US18164348

申请日：2023-02-03

Applicant: Adobe Inc.

Inventor： Xiaojuan Wang ,  Richard Zhang ,  Taesung Park ,  Yang Zhou ,  Elya Shechtman

IPC: H04N21/234 ,  G06V10/771 ,  G06V10/82 ,  H04N21/81

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize machine learning to generate a sequence of transition frames for a gap in a clipped digital video. For example, the disclosed system receives a clipped digital video that includes a pre-cut frame prior to a gap in the clipped digital video and a post-cut frame following the gap in the clipped digital video. Moreover, the disclosed system utilizes a natural motion sequence model to generates a sequence of transition keypoint maps between the pre-cut frame and the post-cut frame. Furthermore, using a generative neural network, the disclosed system generates a sequence of transition frames for the gap in the clipped digital video from the sequence of transition keypoint maps.

公开(公告)号：US20240267597A1

公开(公告)日：2024-08-08

申请号：US18164348

申请日：2023-02-03

Applicant: Adobe Inc.

Inventor： Xiaojuan Wang ,  Richard Zhang ,  Taesung Park ,  Yang Zhou ,  Elya Shechtman

IPC: H04N21/81 ,  G06V10/771 ,  G06V10/82 ,  H04N21/234

CPC classification number: H04N21/8153 ,  G06V10/771 ,  G06V10/82 ,  H04N21/23424

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize machine learning to generate a sequence of transition frames for a gap in a clipped digital video. For example, the disclosed system receives a clipped digital video that includes a pre-cut frame prior to a gap in the clipped digital video and a post-cut frame following the gap in the clipped digital video. Moreover, the disclosed system utilizes a natural motion sequence model to generates a sequence of transition keypoint maps between the pre-cut frame and the post-cut frame. Furthermore, using a generative neural network, the disclosed system generates a sequence of transition frames for the gap in the clipped digital video from the sequence of transition keypoint maps.

公开(公告)号：US20240169604A1

公开(公告)日：2024-05-23

申请号：US18057453

申请日：2022-11-21

Applicant: ADOBE INC.

Inventor： Yosef Gandelsman ,  Taesung Park ,  Richard Zhang ,  Elya Shechtman ,  Alexei A. Efros

IPC: G06T11/00 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/20

CPC classification number: G06T11/001 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/20

Abstract: Systems and methods for image generation are described. Embodiments of the present disclosure obtain user input that indicates a target color and a semantic label for a region of an image to be generated. The system also generates of obtains a noise map including noise biased towards the target color in the region indicated by the user input. A diffusion model generates the image based on the noise map and the semantic label for the region. The image can include an object in the designated region that is described by the semantic label and that has the target color.

公开(公告)号：US20240087265A1

公开(公告)日：2024-03-14

申请号：US17942101

申请日：2022-09-09

Applicant: ADOBE INC.

Inventor： Taesung Park ,  Richard Zhang ,  Elya Schechtman

IPC: G06T19/20 ,  G06F3/04847 ,  G06F40/289 ,  G06V10/774 ,  G06V10/776 ,  G06V10/82

CPC classification number: G06T19/20 ,  G06F3/04847 ,  G06F40/289 ,  G06V10/774 ,  G06V10/776 ,  G06V10/82 ,  G06F40/284 ,  G06T2200/24 ,  G06T2210/61

Abstract: Various disclosed embodiments are directed to changing parameters of an input image or multidimensional representation of the input image based on a user request to change such parameters. An input image is first received. A multidimensional image that represents the input image in multiple dimensions is generated via a model. A request to change at least a first parameter to a second parameter is received via user input at a user device. Such request is a request to edit or generate the multidimensional image in some way. For instance, the request may be to change the light source position or camera position from a first set of coordinates to a second set of coordinates.

公开(公告)号：US11893763B2

公开(公告)日：2024-02-06

申请号：US18058163

申请日：2022-11-22

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Richard Zhang ,  Oliver Wang ,  Junyan Zhu ,  Jingwan Lu ,  Elya Shechtman ,  Alexei A Efros

IPC: G06T9/00 ,  G06N3/08 ,  G06T3/40

CPC classification number: G06T9/002 ,  G06N3/08 ,  G06T3/4046 ,  G06T2200/24 ,  G06T2210/36 ,  G06T2219/2024

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.

公开(公告)号：US20210358177A1

公开(公告)日：2021-11-18

申请号：US16874399

申请日：2020-05-14

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Richard Zhang ,  Oliver Wang ,  Junyan Zhu ,  Jingwan Lu ,  Elya Shechtman ,  Alexei A Efros

IPC: G06T9/00 ,  G06T3/40 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.