公开(公告)号：US12086965B2

公开(公告)日：2024-09-10

申请号：US17520361

申请日：2021-11-05

Applicant: Adobe Inc.

Inventor： Yunhan Zhao ,  Connelly Barnes ,  Yuqian Zhou ,  Sohrab Amirghodsi ,  Elya Shechtman

IPC: G06T5/77 ,  G06T3/18 ,  G06T3/4046 ,  G06T5/50 ,  G06T7/30 ,  G06T7/50 ,  G06T7/90

CPC classification number: G06T5/77 ,  G06T3/18 ,  G06T3/4046 ,  G06T5/50 ,  G06T7/30 ,  G06T7/50 ,  G06T7/90 ,  G06T2207/20084 ,  G06T2207/20221

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media for accurately restoring missing pixels within a hole region of a target image utilizing multi-image inpainting techniques based on incorporating geometric depth information. For example, in various implementations, the disclosed systems utilize a depth prediction of a source image as well as camera relative pose parameters. Additionally, in some implementations, the disclosed systems jointly optimize the depth rescaling and camera pose parameters before generating the reprojected image to further increase the accuracy of the reprojected image. Further, in various implementations, the disclosed systems utilize the reprojected image in connection with a content-aware fill model to generate a refined composite image that includes the target image having a hole, where the hole is filled in based on the reprojected image of the source image.

公开(公告)号：US20240267597A1

公开(公告)日：2024-08-08

申请号：US18164348

申请日：2023-02-03

Applicant: Adobe Inc.

Inventor： Xiaojuan Wang ,  Richard Zhang ,  Taesung Park ,  Yang Zhou ,  Elya Shechtman

IPC: H04N21/81 ,  G06V10/771 ,  G06V10/82 ,  H04N21/234

CPC classification number: H04N21/8153 ,  G06V10/771 ,  G06V10/82 ,  H04N21/23424

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize machine learning to generate a sequence of transition frames for a gap in a clipped digital video. For example, the disclosed system receives a clipped digital video that includes a pre-cut frame prior to a gap in the clipped digital video and a post-cut frame following the gap in the clipped digital video. Moreover, the disclosed system utilizes a natural motion sequence model to generates a sequence of transition keypoint maps between the pre-cut frame and the post-cut frame. Furthermore, using a generative neural network, the disclosed system generates a sequence of transition frames for the gap in the clipped digital video from the sequence of transition keypoint maps.

公开(公告)号：US12056849B2

公开(公告)日：2024-08-06

申请号：US17466711

申请日：2021-09-03

Applicant: Adobe Inc. ,  Czech Technical University in Prague

Inventor： Michal Lukác ,  Daniel Sýkora ,  David Futschik ,  Zhaowen Wang ,  Elya Shechtman

IPC: G06T5/50 ,  G06F18/214

CPC classification number: G06T5/50 ,  G06F18/214 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Embodiments are disclosed for translating an image from a source visual domain to a target visual domain. In particular, in one or more embodiments, the disclosed systems and methods comprise a training process that includes receiving a training input including a pair of keyframes and an unpaired image. The pair of keyframes represent a visual translation from a first version of an image in a source visual domain to a second version of the image in a target visual domain. The one or more embodiments further include sending the pair of keyframes and the unpaired image to an image translation network to generate a first training image and a second training image. The one or more embodiments further include training the image translation network to translate images from the source visual domain to the target visual domain based on a calculated loss using the first and second training images.

公开(公告)号：US20240169604A1

公开(公告)日：2024-05-23

申请号：US18057453

申请日：2022-11-21

Applicant: ADOBE INC.

Inventor： Yosef Gandelsman ,  Taesung Park ,  Richard Zhang ,  Elya Shechtman ,  Alexei A. Efros

IPC: G06T11/00 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/20

CPC classification number: G06T11/001 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/20

Abstract: Systems and methods for image generation are described. Embodiments of the present disclosure obtain user input that indicates a target color and a semantic label for a region of an image to be generated. The system also generates of obtains a noise map including noise biased towards the target color in the region indicated by the user input. A diffusion model generates the image based on the noise map and the semantic label for the region. The image can include an object in the designated region that is described by the semantic label and that has the target color.

公开(公告)号：US11983628B2

公开(公告)日：2024-05-14

申请号：US17468487

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Wei-An Lin ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Jun-Yan Zhu ,  Niloy Mitra ,  Ratheesh Kalarot ,  Richard Zhang ,  Shabnam Ghadar ,  Zhixin Shu

IPC: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/21 ,  G06F18/211 ,  G06F18/214 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/02 ,  G06T3/18 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/20 ,  G06T5/77 ,  G06T11/00 ,  G06T11/60

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/02 ,  G06T3/18 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/20 ,  G06T5/77 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: Systems and methods dynamically adjust an available range for editing an attribute in an image. An image editing system computes a metric for an attribute in an input image as a function of a latent space representation of the input image and a filtering vector for editing the input image. The image editing system compares the metric to a threshold. If the metric exceeds the threshold, then the image editing system selects a first range for editing the attribute in the input image. If the metric does not exceed the threshold, a second range is selected. The image editing system causes display of a user interface for editing the input image comprising an interface element for editing the attribute within the selected range.

公开(公告)号：US20240127452A1

公开(公告)日：2024-04-18

申请号：US17937680

申请日：2022-10-03

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Haitian Zheng ,  Elya Shechtman ,  Jianming Zhang ,  Jingwan Lu ,  Ning Xu ,  Qing Liu ,  Scott Cohen ,  Sohrab Amirghodsi

IPC: G06T7/11

CPC classification number: G06T7/11 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20132

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

公开(公告)号：US11948281B2

公开(公告)日：2024-04-02

申请号：US16864388

申请日：2020-05-01

Applicant: ADOBE INC.

Inventor： Zhe Lin ,  Yu Zeng ,  Jimei Yang ,  Jianming Zhang ,  Elya Shechtman

IPC: G06T5/00 ,  G06T3/40 ,  G06T3/4046 ,  G06T3/4076

CPC classification number: G06T5/005 ,  G06T3/4046 ,  G06T3/4076

Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of high-resolution images using guided upsampling during image inpainting. For instance, an image inpainting system can apply guided upsampling to an inpainted image result to enable generation of a high-resolution inpainting result from a lower-resolution image that has undergone inpainting. To allow for guided upsampling during image inpainting, one or more neural networks can be used. For instance, a low-resolution result neural network (e.g., comprised of an encoder and a decoder) and a high-resolution input neural network (e.g., comprised of an encoder and a decoder). The image inpainting system can use such networks to generate a high-resolution inpainting image result that fills the hole, region, and/or portion of the image.

公开(公告)号：US11893763B2

公开(公告)日：2024-02-06

申请号：US18058163

申请日：2022-11-22

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Richard Zhang ,  Oliver Wang ,  Junyan Zhu ,  Jingwan Lu ,  Elya Shechtman ,  Alexei A Efros

IPC: G06T9/00 ,  G06N3/08 ,  G06T3/40

CPC classification number: G06T9/002 ,  G06N3/08 ,  G06T3/4046 ,  G06T2200/24 ,  G06T2210/36 ,  G06T2219/2024

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.

公开(公告)号：US20230342893A1

公开(公告)日：2023-10-26

申请号：US17660090

申请日：2022-04-21

Applicant: Adobe Inc.

Inventor： Tobias Hinz ,  Shabnam Ghadar ,  Richard Zhang ,  Ratheesh Kalarot ,  Jingwan Lu ,  Elya Shechtman

IPC: G06T5/50 ,  G06T11/60 ,  G06V10/82

CPC classification number: G06T5/50 ,  G06T11/60 ,  G06V10/82 ,  G06T2207/20221 ,  G06T2207/30201

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for combining digital images. In particular, in one or more embodiments, the disclosed systems combine latent codes of a source digital image and a target digital image utilizing a blending network to determine a combined latent encoding and generate a combined digital image from the combined latent encoding utilizing a generative neural network. In some embodiments, the disclosed systems determine an intersection face mask between the source digital image and the combined digital image utilizing a face segmentation network and combine the source digital image and the combined digital image utilizing the intersection face mask to generate a blended digital image.

公开(公告)号：US11769227B2

公开(公告)日：2023-09-26

申请号：US17400426

申请日：2021-08-12

Applicant: Adobe Inc.

Inventor： Yuheng Li ,  Yijun Li ,  Jingwan Lu ,  Elya Shechtman ,  Krishna Kumar Singh

IPC: G06K9/00 ,  G06T3/40 ,  G06N3/04 ,  G06V10/40 ,  G06V30/262 ,  G06F18/25

CPC classification number: G06T3/4046 ,  G06F18/253 ,  G06N3/04 ,  G06V10/40 ,  G06V30/274

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate synthetized digital images via multi-resolution generator neural networks. The disclosed system extracts multi-resolution features from a scene representation to condition a spatial feature tensor and a latent code to modulate an output of a generator neural network. For example, the disclosed systems utilizes a base encoder of the generator neural network to generate a feature set from a semantic label map of a scene. The disclosed system then utilizes a bottom-up encoder to extract multi-resolution features and generate a latent code from the feature set. Furthermore, the disclosed system determines a spatial feature tensor by utilizing a top-down encoder to up-sample and aggregate the multi-resolution features. The disclosed system then utilizes a decoder to generate a synthesized digital image based on the spatial feature tensor and the latent code.