公开(公告)号：US12136199B2

公开(公告)日：2024-11-05

申请号：US17350136

申请日：2021-06-17

Applicant: ADOBE INC.

Inventor： Sohrab Amirghodsi ,  Elya Shechtman ,  Derek Novo

IPC: G06T5/77 ,  G06T5/70 ,  G06T7/194 ,  G06T7/70 ,  G06T7/90 ,  G06T5/20

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for automatically synthesizing a content-aware sampling region for a hole-filling algorithm such as content-aware fill. Given a source image and a hole (or other target region to fill), a sampling region can be synthesized by identifying a band of pixels surrounding the hole, clustering these pixels based on one or more characteristics (e.g., color, x/y coordinates, depth, focus, etc.), passing each of the resulting clusters as foreground pixels to a segmentation algorithm, and unioning the resulting pixels to form the sampling region. The sampling region can be stored in a constraint mask and passed to a hole-filling algorithm such as content-aware fill to synthesize a fill for the hole (or other target region) from patches sampled from the synthesized sampling region.

公开(公告)号：US20240338869A1

公开(公告)日：2024-10-10

申请号：US18474536

申请日：2023-09-26

Applicant: ADOBE INC.

Inventor： Yuqian Zhou ,  Krishna Kumar Singh ,  Zhifei Zhang ,  Difan Liu ,  Zhe Lin ,  Jianming Zhang ,  Qing Liu ,  Jingwan Lu ,  Elya Shechtman ,  Sohrab Amirghodsi ,  Connelly Stuart Barnes

IPC: G06T11/60

CPC classification number: G06T11/60

Abstract: An image processing system obtains an input image (e.g., a user provided image, etc.) and a mask indicating an edit region of the image. A user selects an image editing mode for an image generation network from a plurality of image editing modes. The image generation network generates an output image using the input image, the mask, and the image editing mode.

公开(公告)号：US20240185588A1

公开(公告)日：2024-06-06

申请号：US18062314

申请日：2022-12-06

Applicant: ADOBE INC.

Inventor： Nupur Kumari ,  Richard Zhang ,  Junyan Zhu ,  Elya Shechtman

IPC: G06V10/778 ,  G06V10/75 ,  G06V10/774

CPC classification number: G06V10/778 ,  G06V10/751 ,  G06V10/774

Abstract: Systems and methods for fine-tuning diffusion models are described. Embodiments of the present disclosure obtain an input text indicating an element to be included in an image; generate a synthetic image depicting the element based on the input text using a diffusion model trained by comparing synthetic images depicting the element to training images depicting elements similar to the element and updating selected parameters corresponding to an attention layer of the diffusion model based on the comparison.

公开(公告)号：US20240169500A1

公开(公告)日：2024-05-23

申请号：US18058027

申请日：2022-11-22

Applicant: ADOBE INC.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jianming Zhang ,  Connelly Stuart Barnes ,  Elya Shechtman ,  Jingwan Lu ,  Qing Liu ,  Sohrab Amirghodsi ,  Yuqian Zhou ,  Scott Cohen

IPC: G06T5/00

CPC classification number: G06T5/005 ,  G06T5/003 ,  G06T2207/20081 ,  G06T2207/20104

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive an image comprising a first region that includes content and a second region to be inpainted. Noise is then added to the image to obtain a noisy image, and a plurality of intermediate output images are generated based on the noisy image using a diffusion model trained using a perceptual loss. The intermediate output images predict a final output image based on a corresponding intermediate noise level of the diffusion model. The diffusion model then generates the final output image based on the intermediate output image. The final output image includes inpainted content in the second region that is consistent with the content in the first region.

公开(公告)号：US20240127412A1

公开(公告)日：2024-04-18

申请号：US17937708

申请日：2022-10-03

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Haitian Zheng ,  Elya Shechtman ,  Jianming Zhang ,  Jingwan Lu ,  Ning Xu ,  Qing Liu ,  Scott Cohen ,  Sohrab Amirghodsi

IPC: G06T5/00 ,  G06T7/11

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06T2207/20084 ,  G06T2207/20092

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

公开(公告)号：US11900519B2

公开(公告)日：2024-02-13

申请号：US17455318

申请日：2021-11-17

Applicant: ADOBE INC.

Inventor： Kevin Duarte ,  Wei-An Lin ,  Ratheesh Kalarot ,  Shabnam Ghadar ,  Jingwan Lu ,  Elya Shechtman ,  John Thomas Nack

IPC: G06T13/40 ,  G06T5/50 ,  G06N3/045

CPC classification number: G06T13/40 ,  G06N3/045 ,  G06T5/50

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure encode features of a source image to obtain a source appearance encoding that represents inherent attributes of a face in the source image; encode features of a target image to obtain a target non-appearance encoding that represents contextual attributes of the target image; combine the source appearance encoding and the target non-appearance encoding to obtain combined image features; and generate a modified target image based on the combined image features, wherein the modified target image includes the inherent attributes of the face in the source image together with the contextual attributes of the target image.

公开(公告)号：US20240046429A1

公开(公告)日：2024-02-08

申请号：US17815418

申请日：2022-07-27

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Lingzhi Zhang ,  Zhe Lin ,  Elya Shechtman ,  Yuqian Zhou ,  Connelly Barnes

IPC: G06T5/00 ,  G06T7/11

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06T2207/20084

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating neural network based perceptual artifact segmentations in synthetic digital image content. The disclosed system utilizing neural networks to detect perceptual artifacts in digital images in connection with generating or modifying digital images. The disclosed system determines a digital image including one or more synthetically modified portions. The disclosed system utilizes an artifact segmentation machine-learning model to detect perceptual artifacts in the synthetically modified portion(s). The artifact segmentation machine-learning model is trained to detect perceptual artifacts based on labeled artifact regions of synthetic training digital images. Additionally, the disclosed system utilizes the artifact segmentation machine-learning model in an iterative inpainting process. The disclosed system utilizes one or more digital image inpainting models to inpaint in a digital image. The disclosed system utilizes the artifact segmentation machine-learning model detect perceptual artifacts in the inpainted portions for additional inpainting iterations.

公开(公告)号：US20240037717A1

公开(公告)日：2024-02-01

申请号：US17815409

申请日：2022-07-27

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Lingzhi Zhang ,  Zhe Lin ,  Elya Shechtman ,  Yuqian Zhou ,  Connelly Barnes

IPC: G06T5/00 ,  G06T7/194

CPC classification number: G06T5/005 ,  G06T7/194 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating neural network based perceptual artifact segmentations in synthetic digital image content. The disclosed system utilizing neural networks to detect perceptual artifacts in digital images in connection with generating or modifying digital images. The disclosed system determines a digital image including one or more synthetically modified portions. The disclosed system utilizes an artifact segmentation machine-learning model to detect perceptual artifacts in the synthetically modified portion(s). The artifact segmentation machine-learning model is trained to detect perceptual artifacts based on labeled artifact regions of synthetic training digital images. Additionally, the disclosed system utilizes the artifact segmentation machine-learning model in an iterative inpainting process. The disclosed system utilizes one or more digital image inpainting models to inpaint in a digital image. The disclosed system utilizes the artifact segmentation machine-learning model detect perceptual artifacts in the inpainted portions for additional inpainting iterations.

公开(公告)号：US11887216B2

公开(公告)日：2024-01-30

申请号：US17455796

申请日：2021-11-19

Applicant: ADOBE INC.

Inventor： Ratheesh Kalarot ,  Timothy M. Converse ,  Shabnam Ghadar ,  John Thomas Nack ,  Jingwan Lu ,  Elya Shechtman ,  Baldo Faieta ,  Akhilesh Kumar

IPC: G06T5/00 ,  G06T11/00 ,  G06N3/08 ,  G06V40/16

CPC classification number: G06T11/00 ,  G06N3/08 ,  G06V40/168 ,  G06V40/172

Abstract: The present disclosure describes systems and methods for image processing. Embodiments of the present disclosure include an image processing apparatus configured to generate modified images (e.g., synthetic faces) by conditionally changing attributes or landmarks of an input image. A machine learning model of the image processing apparatus encodes the input image to obtain a joint conditional vector that represents attributes and landmarks of the input image in a vector space. The joint conditional vector is then modified, according to the techniques described herein, to form a latent vector used to generate a modified image. In some cases, the machine learning model is trained using a generative adversarial network (GAN) with a normalization technique, followed by joint training of a landmark embedding and attribute embedding (e.g., to reduce inference time).

公开(公告)号：US11869173B2

公开(公告)日：2024-01-09

申请号：US18089218

申请日：2022-12-27

Applicant: Adobe Inc.

Inventor： Yuqian Zhou ,  Elya Shechtman ,  Connelly Stuart Barnes ,  Sohrab Amirghodsi

IPC: G06T5/00 ,  G06T5/50 ,  G06T3/00 ,  G06N3/08

CPC classification number: G06T5/005 ,  G06N3/08 ,  G06T3/0093 ,  G06T5/50 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2207/20224

Abstract: Various disclosed embodiments are directed to inpainting one or more portions of a target image based on merging (or selecting) one or more portions of a warped image with (or from) one or more portions of an inpainting candidate (e.g., via a learning model). This, among other functionality described herein, resolves the inaccuracies of existing image inpainting technologies.