公开(公告)号：US12008464B2

公开(公告)日：2024-06-11

申请号：US15815635

申请日：2017-11-16

Applicant: ADOBE INC.

Inventor： Haoxiang Li ,  Zhe Lin ,  Jonathan Brandt ,  Xiaohui Shen

IPC: G06N3/08 ,  G06F3/04812 ,  G06F18/2413 ,  G06N3/045 ,  G06T15/04 ,  G06T15/20 ,  G06V10/44 ,  G06V10/764 ,  G06V10/82 ,  G06V40/16

CPC classification number: G06N3/08 ,  G06F3/04812 ,  G06F18/24143 ,  G06N3/045 ,  G06T15/04 ,  G06T15/205 ,  G06V10/454 ,  G06V10/764 ,  G06V10/82 ,  G06V40/165 ,  G06V40/171

Abstract: Approaches are described for determining facial landmarks in images. An input image is provided to at least one trained neural network that determines a face region (e.g., bounding box of a face) of the input image and initial facial landmark locations corresponding to the face region. The initial facial landmark locations are provided to a 3D face mapper that maps the initial facial landmark locations to a 3D face model. A set of facial landmark locations are determined from the 3D face model. The set of facial landmark locations are provided to a landmark location adjuster that adjusts positions of the set of facial landmark locations based on the input image. The input image is presented on a user device using the adjusted set of facial landmark locations.

公开(公告)号：US20240169685A1

公开(公告)日：2024-05-23

申请号：US18058575

申请日：2022-11-23

Applicant: Adobe Inc.

Inventor： Luis Figueroa ,  Zhe Lin ,  Zhihong Ding ,  Scott Cohen

IPC: G06V10/20 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/60 ,  G06V10/82

CPC classification number: G06V10/255 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/60 ,  G06V10/82

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems receive a digital image from a client device. The disclosed systems detect, utilizing a shadow detection neural network, an object portrayed in the digital image. The disclosed systems detect, utilizing the shadow detection neural network, a shadow portrayed in the digital image. The disclosed systems generate, utilizing the shadow detection neural network, an object-shadow pair prediction that associates the shadow with the object.

公开(公告)号：US20240169500A1

公开(公告)日：2024-05-23

申请号：US18058027

申请日：2022-11-22

Applicant: ADOBE INC.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jianming Zhang ,  Connelly Stuart Barnes ,  Elya Shechtman ,  Jingwan Lu ,  Qing Liu ,  Sohrab Amirghodsi ,  Yuqian Zhou ,  Scott Cohen

IPC: G06T5/00

CPC classification number: G06T5/005 ,  G06T5/003 ,  G06T2207/20081 ,  G06T2207/20104

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive an image comprising a first region that includes content and a second region to be inpainted. Noise is then added to the image to obtain a noisy image, and a plurality of intermediate output images are generated based on the noisy image using a diffusion model trained using a perceptual loss. The intermediate output images predict a final output image based on a corresponding intermediate noise level of the diffusion model. The diffusion model then generates the final output image based on the intermediate output image. The final output image includes inpainted content in the second region that is consistent with the content in the first region.

公开(公告)号：US11983632B2

公开(公告)日：2024-05-14

申请号：US18309367

申请日：2023-04-28

Applicant: Adobe Inc.

Inventor： Shikun Liu ,  Zhe Lin ,  Yilin Wang ,  Jianming Zhang ,  Federico Perazzi

IPC: G06N3/082 ,  G06N3/04

CPC classification number: G06N3/082 ,  G06N3/04

Abstract: The disclosure describes one or more implementations of a neural network architecture pruning system that automatically and progressively prunes neural networks. For instance, the neural network architecture pruning system can automatically reduce the size of an untrained or previously-trained neural network without reducing the accuracy of the neural network. For example, the neural network architecture pruning system jointly trains portions of a neural network while progressively pruning redundant subsets of the neural network at each training iteration. In many instances, the neural network architecture pruning system increases the accuracy of the neural network by progressively removing excess or redundant portions (e.g., channels or layers) of the neural network. Further, by removing portions of a neural network, the neural network architecture pruning system can increase the efficiency of the neural network.

公开(公告)号：US20240135514A1

公开(公告)日：2024-04-25

申请号：US18460365

申请日：2023-09-01

Applicant: Adobe Inc.

Inventor： Daniil Pakhomov ,  Qing Liu ,  Zhihong Ding ,  Scott Cohen ,  Zhe Lin ,  Jianming Zhang ,  Zhifei Zhang ,  Ohiremen Dibua ,  Mariette Souppe ,  Krishna Kumar Singh ,  Jonathan Brandt

IPC: G06T5/00 ,  G06F3/04845 ,  G06T7/11 ,  G06T7/194 ,  G06T7/70

CPC classification number: G06T5/005 ,  G06F3/04845 ,  G06T5/002 ,  G06T7/11 ,  G06T7/194 ,  G06T7/70 ,  G06T2200/24 ,  G06T2207/20021 ,  G06T2207/20084 ,  G06T2207/20092

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via multi-layered scene completion techniques facilitated by artificial intelligence. For instance, in some embodiments, the disclosed systems receive a digital image portraying a first object and a second object against a background, where the first object occludes a portion of the second object. Additionally, the disclosed systems pre-process the digital image to generate a first content fill for the portion of the second object occluded by the first object and a second content fill for a portion of the background occluded by the second object. After pre-processing, the disclosed systems detect one or more user interactions to move or delete the first object from the digital image. The disclosed systems further modify the digital image by moving or deleting the first object and exposing the first content fill for the portion of the second object.

公开(公告)号：US20240127412A1

公开(公告)日：2024-04-18

申请号：US17937708

申请日：2022-10-03

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Haitian Zheng ,  Elya Shechtman ,  Jianming Zhang ,  Jingwan Lu ,  Ning Xu ,  Qing Liu ,  Scott Cohen ,  Sohrab Amirghodsi

IPC: G06T5/00 ,  G06T7/11

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06T2207/20084 ,  G06T2207/20092

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

公开(公告)号：US11960843B2

公开(公告)日：2024-04-16

申请号：US16401548

申请日：2019-05-02

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Trung Huu Bui ,  Scott Cohen ,  Mingyang Ling ,  Chenyun Wu

IPC: G06N20/00 ,  G06F40/30 ,  G06V10/25 ,  G06V10/764 ,  G06V10/82 ,  G06F18/21 ,  G06F40/205

CPC classification number: G06F40/30 ,  G06N20/00 ,  G06V10/25 ,  G06V10/764 ,  G06V10/82 ,  G06F18/217 ,  G06F40/205

Abstract: Techniques and systems are provided for training a machine learning model using different datasets to perform one or more tasks. The machine learning model can include a first sub-module configured to perform a first task and a second sub-module configured to perform a second task. The first sub-module can be selected for training using a first training dataset based on a format of the first training dataset. The first sub-module can then be trained using the first training dataset to perform the first task. The second sub-module can be selected for training using a second training dataset based on a format of the second training dataset. The second sub-module can then be trained using the second training dataset to perform the second task.

公开(公告)号：US11941884B2

公开(公告)日：2024-03-26

申请号：US17454740

申请日：2021-11-12

Applicant: ADOBE INC.

Inventor： Jason Wen Yong Kuen ,  Bo Sun ,  Zhe Lin ,  Simon Su Chen

IPC: G06K9/00 ,  G06F18/21 ,  G06N3/08 ,  G06T9/00 ,  G06V10/75 ,  G06V20/40

CPC classification number: G06V20/41 ,  G06F18/2163 ,  G06N3/08 ,  G06T3/4046 ,  G06T9/002 ,  G06V10/751

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive an image having a plurality of object instances; encode the image to obtain image features; decode the image features to obtain object features; generate object detection information based on the object features using an object detection branch, wherein the object detection branch is trained based on a first training set using a detection loss; generate semantic segmentation information based on the object features using a semantic segmentation branch, wherein the semantic segmentation branch is trained based on a second training set different from the first training set using a semantic segmentation loss; and combine the object detection information and the semantic segmentation information to obtain panoptic segmentation information that indicates which pixels of the image correspond to each of the plurality of object instances.

公开(公告)号：US20240046429A1

公开(公告)日：2024-02-08

申请号：US17815418

申请日：2022-07-27

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Lingzhi Zhang ,  Zhe Lin ,  Elya Shechtman ,  Yuqian Zhou ,  Connelly Barnes

IPC: G06T5/00 ,  G06T7/11

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06T2207/20084

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating neural network based perceptual artifact segmentations in synthetic digital image content. The disclosed system utilizing neural networks to detect perceptual artifacts in digital images in connection with generating or modifying digital images. The disclosed system determines a digital image including one or more synthetically modified portions. The disclosed system utilizes an artifact segmentation machine-learning model to detect perceptual artifacts in the synthetically modified portion(s). The artifact segmentation machine-learning model is trained to detect perceptual artifacts based on labeled artifact regions of synthetic training digital images. Additionally, the disclosed system utilizes the artifact segmentation machine-learning model in an iterative inpainting process. The disclosed system utilizes one or more digital image inpainting models to inpaint in a digital image. The disclosed system utilizes the artifact segmentation machine-learning model detect perceptual artifacts in the inpainted portions for additional inpainting iterations.

公开(公告)号：US20240037717A1

公开(公告)日：2024-02-01

申请号：US17815409

申请日：2022-07-27

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Lingzhi Zhang ,  Zhe Lin ,  Elya Shechtman ,  Yuqian Zhou ,  Connelly Barnes

IPC: G06T5/00 ,  G06T7/194

CPC classification number: G06T5/005 ,  G06T7/194 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating neural network based perceptual artifact segmentations in synthetic digital image content. The disclosed system utilizing neural networks to detect perceptual artifacts in digital images in connection with generating or modifying digital images. The disclosed system determines a digital image including one or more synthetically modified portions. The disclosed system utilizes an artifact segmentation machine-learning model to detect perceptual artifacts in the synthetically modified portion(s). The artifact segmentation machine-learning model is trained to detect perceptual artifacts based on labeled artifact regions of synthetic training digital images. Additionally, the disclosed system utilizes the artifact segmentation machine-learning model in an iterative inpainting process. The disclosed system utilizes one or more digital image inpainting models to inpaint in a digital image. The disclosed system utilizes the artifact segmentation machine-learning model detect perceptual artifacts in the inpainted portions for additional inpainting iterations.