公开(公告)号：US20220284236A1

公开(公告)日：2022-09-08

申请号：US17190197

申请日：2021-03-02

Applicant: ADOBE INC.

Inventor： Aashish Misraa ,  Zhe Lin

IPC: G06K9/62 ,  G06T7/00

Abstract: Systems and methods for image processing are described. Embodiments identify a training set including a first image that includes a ground truth blur classification and second image that includes a ground truth blur map, generate a first embedded representation of the first image and a second embedded representation of the second image using an image encoder, predict a blur classification of the first image based on the first embedded representation using a classification layer, predict a blur map of the second image based on the second embedded representation using a map decoder, compute a classification loss based on the predicted blur classification and the ground truth blur classification, train the image encoder and the classification layer based on the classification loss, compute a map loss based on the blur map and the ground truth blur map, and train the image encoder and the map decoder.

公开(公告)号：US11436775B2

公开(公告)日：2022-09-06

申请号：US16806344

申请日：2020-03-02

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xin Lu ,  Xiaohui Shen ,  Jimei Yang ,  Jiahui Yu

IPC: G06T11/60 ,  G06K9/62 ,  G06T5/00 ,  G06T5/30 ,  G06N3/04 ,  G06N3/08 ,  G06V10/44 ,  G06V10/75 ,  G06V30/194

Abstract: Predicting patch displacement maps using a neural network is described. Initially, a digital image on which an image editing operation is to be performed is provided as input to a patch matcher having an offset prediction neural network. From this image and based on the image editing operation for which this network is trained, the offset prediction neural network generates an offset prediction formed as a displacement map, which has offset vectors that represent a displacement of pixels of the digital image to different locations for performing the image editing operation. Pixel values of the digital image are copied to the image pixels affected by the operation.

公开(公告)号：US20220262011A1

公开(公告)日：2022-08-18

申请号：US17735728

申请日：2022-05-03

Applicant: ADOBE INC.

Inventor： Lu Zhang ,  Jianming Zhang ,  Zhe Lin ,  Radomir Mech

IPC: G06T7/20 ,  G06T7/00 ,  G06T11/60 ,  G06F3/01 ,  G06N3/02 ,  G06F3/04845

Abstract: Systems and methods provide editing operations in a smart editing system that may generate a focal point within a mask of an object for each frame of a video segment and perform editing effects on the frames of the video segment to quickly provide users with natural video editing effects. An eye-gaze network may produce a hotspot map of predicted focal points in a video frame. These predicted focal points may then be used by a gaze-to-mask network to determine objects in the image and generate an object mask for each of the detected objects. This process may then be repeated to effectively track the trajectory of objects and object focal points in videos. Based on the determined trajectory of an object in a video clip and editing parameters, the editing engine may produce editing effects relative to an object for the video clip.

公开(公告)号：US11410038B2

公开(公告)日：2022-08-09

申请号：US17204370

申请日：2021-03-17

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Radomir Mech ,  Jian Ren

IPC: G06N3/08 ,  G06K9/62 ,  G06F16/783 ,  G06N3/04 ,  G06V20/40 ,  G06V40/16

Abstract: Various embodiments describe frame selection based on training and using a neural network. In an example, the neural network is a convolutional neural network trained with training pairs. Each training pair includes two training frames from a frame collection. The loss function relies on the estimated quality difference between the two training frames. Further, the definition of the loss function varies based on the actual quality difference between these two frames. In a further example, the neural network is trained by incorporating facial heatmaps generated from the training frames and facial quality scores of faces detected in the training frames. In addition, the training involves using a feature mean that represents an average of the features of the training frames belonging to the same frame collection. Once the neural network is trained, a frame collection is input thereto and a frame is selected based on generated quality scores.

公开(公告)号：US20220237682A1

公开(公告)日：2022-07-28

申请号：US17159554

申请日：2021-01-27

Applicant: ADOBE INC.

Inventor： Handong Zhao ,  Zhankui He ,  Zhaowen Wang ,  Zhe Lin ,  Ajinkya Kale ,  Fengbin Chen

IPC: G06Q30/06 ,  G06N3/04 ,  G06N3/08 ,  G06K9/62

Abstract: Systems and methods for item recommendation are described. Embodiments identify a sequence of items selected by a user, embed each item of the sequence of items to produce item embeddings having a reduced number of dimensions, predict a next item based on the item embeddings using a recommendation network, wherein the recommendation network includes a sequential encoder trained based at least in part on a sampled softmax classifier, and wherein predicting the next item represents a prediction that the user will interact with the next item, and provide a recommendation to the user, wherein the recommendation includes the next item.

公开(公告)号：US20220222532A1

公开(公告)日：2022-07-14

申请号：US17147912

申请日：2021-01-13

Applicant: Adobe Inc.

Inventor： Zhixin Shu ,  Zhe Lin ,  Yuchen Liu ,  Yijun Li

IPC: G06N3/08 ,  G06T5/00 ,  G06T5/40 ,  G06T7/194

Abstract: This disclosure describes one or more embodiments of systems, non-transitory computer-readable media, and methods that utilize channel pruning and knowledge distillation to generate a compact noise-to-image GAN. For example, the disclosed systems prune less informative channels via outgoing channel weights of the GAN. In some implementations, the disclosed systems further utilize content-aware pruning by utilizing a differentiable loss between an image generated by the GAN and a modified version of the image to identify sensitive channels within the GAN during channel pruning. In some embodiments, the disclosed systems utilize knowledge distillation to learn parameters for the pruned GAN to mimic a full-size GAN. In certain implementations, the disclosed systems utilize content-aware knowledge distillation by applying content masks on images generated by both the pruned GAN and its full-size counterpart to obtain knowledge distillation losses between the images for use in learning the parameters for the pruned GAN.

公开(公告)号：US11380033B2

公开(公告)日：2022-07-05

申请号：US16738359

申请日：2020-01-09

Applicant: Adobe Inc.

Inventor： Kate Sousa ,  Zhe Lin ,  Saeid Motiian ,  Pramod Srinivasan ,  Baldo Faieta ,  Alex Filipkowski

IPC: G06T11/60 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08 ,  G06V10/25

Abstract: Based on a received digital image and text, a neural network trained to identify candidate text placement areas within images may be used to generate a mask for the digital image that includes a candidate text placement area. A bounding box for the digital image may be defined for the text and based on the candidate text placement area, and the text may be superimposed onto the digital image within the bounding box.

公开(公告)号：US11367273B2

公开(公告)日：2022-06-21

申请号：US16919383

申请日：2020-07-02

Applicant: Adobe Inc.

Inventor： Delun Du ,  Zhe Lin ,  Baldo Faieta

IPC: G06N3/08 ,  G06K9/62 ,  G06K9/20 ,  G06N3/04 ,  G06V10/50 ,  G06V10/22

Abstract: The present disclosure is directed toward systems and methods for detecting an object in an input image based on a target object keyword. For example, one or more embodiments described herein generate a heat map of the input image based on the target object keyword and generate various bounding boxes based on a pixel analysis of the heat map. One or more embodiments described herein then utilize the various bounding boxes to determine scores for generated object location proposals in order to provide a highest scoring object location proposal overlaid on the input image.

公开(公告)号：US11321847B2

公开(公告)日：2022-05-03

申请号：US17103119

申请日：2020-11-24

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Wei Xiong ,  Connelly Barnes ,  Jimei Yang ,  Xin Lu

IPC: G06T7/149 ,  G06T7/13 ,  G06T7/194 ,  G06T5/00

Abstract: In some embodiments, an image manipulation application receives an incomplete image that includes a hole area lacking image content. The image manipulation application applies a contour detection operation to the incomplete image to detect an incomplete contour of a foreground object in the incomplete image. The hole area prevents the contour detection operation from detecting a completed contour of the foreground object. The image manipulation application further applies a contour completion model to the incomplete contour and the incomplete image to generate the completed contour for the foreground object. Based on the completed contour and the incomplete image, the image manipulation application generates image content for the hole area to generate a completed image.

公开(公告)号：US20220121702A1

公开(公告)日：2022-04-21

申请号：US17075450

申请日：2020-10-20

Applicant: Adobe Inc.

Inventor： Ajinkya Kale ,  Zhe Lin ,  Pranav Aggarwal

IPC: G06F16/535 ,  G06K9/62 ,  G06F40/279 ,  G06F16/242 ,  G06F16/538 ,  G06N3/04 ,  G06N3/08

Abstract: The present disclosure relates to methods, systems, and non-transitory computer-readable media for retrieving digital images in response to queries. For example, in one or more embodiments, the disclosed systems receive a query comprising text and generates a cross-lingual-multimodal embedding for the text within a multimodal embedding space. The disclosed systems further identifies an image embedding for a digital image that corresponds to (e.g., is relevant to) the text from the query based on an embedding distance between the image embedding and the cross-lingual-multimodal embedding for the text within the multimodal embedding space. Accordingly, the disclosed systems retrieve the digital image associated with the image embedding for display on a client device, such as the client device that submitted the query.