公开(公告)号：US11868889B2

公开(公告)日：2024-01-09

申请号：US17588516

申请日：2022-01-31

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Mingyang Ling ,  Jianming Zhang ,  Jason Wen Yong Kuen

IPC: G06N3/08 ,  G06N3/04 ,  G06V20/20 ,  G06V20/64 ,  G06V10/82 ,  G06V20/10 ,  G06F18/214 ,  G06V10/764 ,  G06V10/44

CPC classification number: G06N3/08 ,  G06F18/214 ,  G06N3/04 ,  G06V10/454 ,  G06V10/764 ,  G06V10/82 ,  G06V20/10 ,  G06V20/20 ,  G06V20/64

Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.

公开(公告)号：US20230418861A1

公开(公告)日：2023-12-28

申请号：US17809503

申请日：2022-06-28

Applicant: Adobe Inc.

Inventor： Zhifei Zhang ,  Zhe Lin

IPC: G06F16/535 ,  G06F16/532 ,  G06F16/33

CPC classification number: G06F16/535 ,  G06F16/532 ,  G06F16/3334

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, in one or more embodiments, the disclosed systems receive an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and the disclosed systems retrieve the image search results utilizing a weighted combination of the queries. In some implementations, the disclosed systems generate an input embedding for the search input (e.g., the multi-modal search input) and retrieve the image search results using the input embedding.

公开(公告)号：US11854119B2

公开(公告)日：2023-12-26

申请号：US17155570

申请日：2021-01-22

Applicant: Adobe Inc.

Inventor： Siavash Khodadadeh ,  Zhe Lin ,  Shabnam Ghadar ,  Saeid Motiian ,  Richard Zhang ,  Ratheesh Kalarot ,  Baldo Faieta

IPC: G06N3/08 ,  G06T7/90 ,  G06N3/045 ,  G06T11/00

CPC classification number: G06T11/001 ,  G06N3/045 ,  G06N3/08 ,  G06T7/90

Abstract: Embodiments are disclosed for automatic object re-colorization in images. In some embodiments, a method of automatic object re-colorization includes receiving a request to recolor an object in an image, the request including an object identifier and a color identifier, identifying an object in the image associated with the object identifier, generating a mask corresponding to the object in the image, providing the image, the mask, and the color identifier to a color transformer network, the color transformer network trained to recolor objects in input images, and generating, by the color transformer network, a recolored image, wherein the object in the recolored image has been recolored to a color corresponding to the color identifier.

公开(公告)号：US20230401827A1

公开(公告)日：2023-12-14

申请号：US17806097

申请日：2022-06-09

Applicant: ADOBE INC.

Inventor： Jason Wen Yong Kuen ,  Dat Ba Huynh ,  Zhe Lin ,  Jiuxiang Gu

IPC: G06V10/774 ,  G06V10/26 ,  G06V10/75 ,  G06V10/77 ,  G06V10/776 ,  G06V10/82

CPC classification number: G06V10/774 ,  G06V10/26 ,  G06V10/759 ,  G06V10/7715 ,  G06V10/776 ,  G06V10/82

Abstract: Systems and methods for image segmentation are described. Embodiments of the present disclosure receive a training image and a caption for the training image, wherein the caption includes text describing an object in the training image; generate a pseudo mask for the object using a teacher network based on the text describing the object; generate a mask for the object using a student network; compute noise information for the training image using a noise estimation network; and update parameters of the student network based on the mask, the pseudo mask, and the noise information.

公开(公告)号：US20230401716A1

公开(公告)日：2023-12-14

申请号：US17806312

申请日：2022-06-10

Applicant: ADOBE INC.

Inventor： Yilin Wang ,  Chenglin Yang ,  Jianming Zhang ,  He Zhang ,  Zijun Wei ,  Zhe Lin

IPC: G06T7/10 ,  G06T11/60 ,  G06N3/08

CPC classification number: G06T7/10 ,  G06T11/60 ,  G06N3/088

Abstract: Systems and methods for image segmentation are described. Embodiments of the present disclosure receive an image depicting an object; generate image features for the image by performing a convolutional self-attention operation that outputs a plurality of attention-weighted values for a convolutional kernel applied at a position of a sliding window on the image; and generate label data that identifies the object based on the image features.

公开(公告)号：US11823490B2

公开(公告)日：2023-11-21

申请号：US17341778

申请日：2021-06-08

Applicant: ADOBE INC.

Inventor： Ratheesh Kalarot ,  Siavash Khodadadeh ,  Baldo Faieta ,  Shabnam Ghadar ,  Saeid Motiian ,  Wei-An Lin ,  Zhe Lin

IPC: G06V40/16 ,  G06N3/084 ,  G06T11/60 ,  G06N3/045

CPC classification number: G06V40/169 ,  G06N3/045 ,  G06N3/084 ,  G06T11/60

Abstract: Systems and methods for image processing are described. One or more embodiments of the present disclosure identify a latent vector representing an image of a face, identify a target attribute vector representing a target attribute for the image, generate a modified latent vector using a mapping network that converts the latent vector and the target attribute vector into a hidden representation having fewer dimensions than the latent vector, wherein the modified latent vector is generated based on the hidden representation, and generate a modified image based on the modified latent vector, wherein the modified image represents the face with the target attribute.

公开(公告)号：US11816181B2

公开(公告)日：2023-11-14

申请号：US17190197

申请日：2021-03-02

Applicant: ADOBE INC.

Inventor： Aashish Misraa ,  Zhe Lin

IPC: G06V10/774 ,  G06F18/214 ,  G06T7/00 ,  G06F18/241 ,  G06V10/82 ,  G06V10/70

CPC classification number: G06F18/214 ,  G06F18/241 ,  G06T7/0002 ,  G06V10/774 ,  G06V10/82 ,  G06V10/87 ,  G06T2207/20081 ,  G06T2207/30168

Abstract: Systems and methods for image processing are described. Embodiments identify a training set including a first image that includes a ground truth blur classification and second image that includes a ground truth blur map, generate a first embedded representation of the first image and a second embedded representation of the second image using an image encoder, predict a blur classification of the first image based on the first embedded representation using a classification layer, predict a blur map of the second image based on the second embedded representation using a map decoder, compute a classification loss based on the predicted blur classification and the ground truth blur classification, train the image encoder and the classification layer based on the classification loss, compute a map loss based on the blur map and the ground truth blur map, and train the image encoder and the map decoder.

公开(公告)号：US11790650B2

公开(公告)日：2023-10-17

申请号：US16998876

申请日：2020-08-20

Applicant: Adobe Inc.

Inventor： Quan Hung Tran ,  Long Thanh Mai ,  Zhe Lin ,  Zhuowan Li

IPC: G06V20/30 ,  G06F16/55 ,  G06F16/535 ,  G06V10/82 ,  G06F40/205 ,  G06V10/75 ,  G06F18/214

CPC classification number: G06V20/30 ,  G06F16/535 ,  G06F16/55 ,  G06F18/214 ,  G06F40/205 ,  G06V10/751 ,  G06V10/82

Abstract: A group captioning system includes computing hardware, software, and/or firmware components in support of the enhanced group captioning contemplated herein. In operation, the system generates a target embedding for a group of target images, as well as a reference embedding for a group of reference images. The system identifies information in-common between the group of target images and the group of reference images and removes the joint information from the target embedding and the reference embedding. The result is a contrastive group embedding that includes a contrastive target embedding and a contrastive reference embedding with which to construct a contrastive group embedding, which is then input to a model to obtain a group caption for the target group of images.

公开(公告)号：US11790234B2

公开(公告)日：2023-10-17

申请号：US18063851

申请日：2022-12-09

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Siyuan Qiao ,  Jianming Zhang

IPC: G06K9/00 ,  G06N3/082 ,  G06N3/04 ,  G06F18/214

CPC classification number: G06N3/082 ,  G06N3/04 ,  G06F18/2148

Abstract: In implementations of resource-aware training for neural network, one or more computing devices of a system implement an architecture optimization module for monitoring parameter utilization while training a neural network. Dead neurons of the neural network are identified as having activation scales less than a threshold. Neurons with activation scales greater than or equal to the threshold are identified as survived neurons. The dead neurons are converted to reborn neurons by adding the dead neurons to layers of the neural network having the survived neurons. The reborn neurons are prevented from connecting to the survived neurons for training the reborn neurons.

公开(公告)号：US11790045B2

公开(公告)日：2023-10-17

申请号：US17240246

申请日：2021-04-26

Applicant: ADOBE INC.

Inventor： Shipali Shetty ,  Zhe Lin ,  Alexander Smith

IPC: G06F18/22 ,  G06T7/00 ,  G06V10/75

CPC classification number: G06F18/22 ,  G06T7/0002 ,  G06V10/75 ,  G06T2207/20132 ,  G06T2210/12

Abstract: Systems and methods for image tagging are described. In some embodiments, images with problematic tags are identified after applying an auto-tagger. The images with problematic tags are then sent to an object detection network. In some cases, the object detection network is trained using a training set selected to improve detection of objects associated with the problematic tags. The output of the object detection network can be merged with the output of the auto-tagger to provide a combined image tagging output. In some cases, the output of the object detection network also includes a bounding box, which can be used to crop the image around a relevant object so that the auto-tagger can be reapplied to a portion of the image.