公开(公告)号：US20210271707A1

公开(公告)日：2021-09-02

申请号：US16803480

申请日：2020-02-27

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06F16/583 ,  G06K9/62 ,  G06K9/72 ,  G06F40/30 ,  G06F16/538 ,  G06F16/56 ,  G06F16/2457 ,  G06N3/08

Abstract: Certain embodiments involve a method for generating a search result. The method includes processing devices performing operations including receiving a query having a text input by a joint embedding model trained to generate an image result. Training the joint embedding model includes accessing a set of images and textual information. Training further includes encoding the images into image feature vectors based on spatial features. Further, training includes encoding the textual information into textual feature vectors based on semantic information. Training further includes generating a set of image-text pairs based on matches between image feature vectors and textual feature vectors. Further, training includes generating a visual grounding dataset based on spatial information. Training further includes generating a set of visual-semantic joint embeddings by grounding the image-text pairs with the visual grounding dataset. Additionally, operations include generating an image result for display by the joint embedding model based on the text input.

公开(公告)号：US20210248177A1

公开(公告)日：2021-08-12

申请号：US16785410

申请日：2020-02-07

Applicant: Adobe Inc.

Inventor： Pramod Srinivasan ,  Zhe Lin ,  Samarth Gulati ,  Saeid Motiian ,  Midhun Harikumar ,  Baldo Antonio Faieta ,  Alex C. Filipkowski

IPC: G06F16/532 ,  G06F40/30 ,  G06F16/51 ,  G06F16/583 ,  G06F16/538 ,  G06F16/54

Abstract: Keyword localization digital image search techniques are described. These techniques support an ability to indicate “where” a corresponding keyword is to be expressed with respect to a layout in a respective digital image resulting from a search query. The search query may also include an indication of a size of the keyword as expressed in the digital image, a number of instances of the keyword, and so forth. Additionally, the techniques and systems as described herein support real time search through use of keyword signatures.

公开(公告)号：US20210217215A1

公开(公告)日：2021-07-15

申请号：US16738359

申请日：2020-01-09

Applicant: Adobe Inc.

Inventor： Kate Sousa ,  Zhe Lin ,  Saeid Motiian ,  Pramod Srinivasan ,  Baldo Faieta ,  Alex Filipkowski

IPC: G06T11/60 ,  G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Based on a received digital image and text, a neural network trained to identify candidate text placement areas within images may be used to generate a mask for the digital image that includes a candidate text placement area. A bounding box for the digital image may be defined for the text and based on the candidate text placement area, and the text may be superimposed onto the digital image within the bounding box.

公开(公告)号：US20210201150A1

公开(公告)日：2021-07-01

申请号：US17204370

申请日：2021-03-17

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Radomir Mech ,  Jian Ren

IPC: G06N3/08 ,  G06K9/62 ,  G06K9/00 ,  G06F16/783 ,  G06N3/04

Abstract: Various embodiments describe frame selection based on training and using a neural network. In an example, the neural network is a convolutional neural network trained with training pairs. Each training pair includes two training frames from a frame collection. The loss function relies on the estimated quality difference between the two training frames. Further, the definition of the loss function varies based on the actual quality difference between these two frames. In a further example, the neural network is trained by incorporating facial heatmaps generated from the training frames and facial quality scores of faces detected in the training frames. In addition, the training involves using a feature mean that represents an average of the features of the training frames belonging to the same frame collection. Once the neural network is trained, a frame collection is input thereto and a frame is selected based on generated quality scores.

公开(公告)号：US10963759B2

公开(公告)日：2021-03-30

申请号：US16417115

申请日：2019-05-20

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Mai Long ,  Jonathan Brandt ,  Hailin Jin ,  Chen Fang

IPC: G06K9/66 ,  G06F16/532 ,  G06K9/46 ,  G06K9/62 ,  G06K9/72 ,  G06N3/04 ,  G06F16/583 ,  G06K9/52 ,  G06N3/08

Abstract: The present disclosure includes methods and systems for searching for digital visual media based on semantic and spatial information. In particular, one or more embodiments of the disclosed systems and methods identify digital visual media displaying targeted visual content in a targeted region based on a query term and a query area provide via a digital canvas. Specifically, the disclosed systems and methods can receive user input of a query term and a query area and provide the query term and query area to a query neural network to generate a query feature set. Moreover, the disclosed systems and methods can compare the query feature set to digital visual media feature sets. Further, based on the comparison, the disclosed systems and methods can identify digital visual media portraying targeted visual content corresponding to the query term within a targeted region corresponding to the query area.

公开(公告)号：US20210064704A1

公开(公告)日：2021-03-04

申请号：US16553305

申请日：2019-08-28

Applicant: Adobe Inc.

Inventor： Yang Yang ,  Zhe Lin

IPC: G06F17/28 ,  G06F16/53 ,  G06F16/58

Abstract: In some embodiments, a context-based translation application generates a co-occurrence data structure for a target language describing co-occurrences of target language words and source language words. The context-based translation application receives an input tag for an input image in the source language to be translated into the target language. The context-based translation application obtains multiple candidate translations in the target language for the input tag and determines a translated tag from the multiple candidate translations based on the co-occurrence data structure. The context-based translation application further associates the translated tag with the input image.

公开(公告)号：US20210027448A1

公开(公告)日：2021-01-28

申请号：US16518850

申请日：2019-07-22

Applicant: Adobe Inc.

Inventor： Scott Cohen ,  Zhe Lin ,  Mingyang Ling

IPC: G06T7/00 ,  G06T7/70 ,  G06T7/90 ,  G06T11/60 ,  G06F16/583 ,  G06F17/27

Abstract: The present disclosure relates to an object selection system that accurately detects and automatically selects target instances of user-requested objects (e.g., a query object instance) in a digital image. In one or more embodiments, the object selection system can analyze one or more user inputs to determine an optimal object attribute detection model from multiple specialized and generalized object attribute models. Additionally, the object selection system can utilize the selected object attribute model to detect and select one or more target instances of a query object in an image, where the image includes multiple instances of the query object.

公开(公告)号：US10885608B2

公开(公告)日：2021-01-05

申请号：US16001656

申请日：2018-06-06

Applicant: Adobe Inc.

Inventor： Zhifei Zhang ,  Zhe Lin ,  Zhaowen Wang

IPC: G06K9/00 ,  G06T3/40 ,  G06K9/62 ,  G06T11/00

Abstract: In implementations of super-resolution with reference images, a super-resolution image is generated based on reference images. Reference images are not constrained to have same or similar content as a low-resolution image being super-resolved. Texture features indicating high-frequency content are extracted into texture feature maps, and patches of texture feature maps of reference images are determined based on texture feature similarity. A content feature map indicating low-frequency content of an image is adaptively fused with a swapped texture feature map including patches of reference images with a neural network based on similarity of texture features. A user interfaces allows a user to select regions of multiple reference images to use for super-resolution. Hence, a super-resolution image can be generated with rich texture details incorporated from multiple reference images, even in the absence of reference images having similar content to an image being upscaled.

公开(公告)号：US20200349464A1

公开(公告)日：2020-11-05

申请号：US16401548

申请日：2019-05-02

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Trung Huu Bui ,  Scott Cohen ,  Mingyang Ling ,  Chenyun Wu

IPC: G06N20/00

Abstract: Techniques and systems are provided for training a machine learning model using different datasets to perform one or more tasks. The machine learning model can include a first sub-module configured to perform a first task and a second sub-module configured to perform a second task. The first sub-module can be selected for training using a first training dataset based on a format of the first training dataset. The first sub-module can then be trained using the first training dataset to perform the first task. The second sub-module can be selected for training using a second training dataset based on a format of the second training dataset. The second sub-module can then be trained using the second training dataset to perform the second task.

公开(公告)号：US20200342255A1

公开(公告)日：2020-10-29

申请号：US16928949

申请日：2020-07-14

Applicant: Adobe Inc.

Inventor： Jayant Kumar ,  Zhe Lin ,  Vipulkumar C. Dalal

IPC: G06K9/62

Abstract: There is described a computing device and method in a digital medium environment for custom auto tagging of multiple objects. The computing device includes an object detection network and multiple image classification networks. An image is received at the object detection network and includes multiple visual objects. First feature maps are applied to the image at the object detection network and generate object regions associated with the visual objects. The object regions are assigned to the multiple image classification networks, and each image classification network is assigned to a particular object region. The second feature maps are applied to each object region at each image classification network, and each image classification network outputs one or more classes associated with a visual object corresponding to each object region.