公开(公告)号：US11949964B2

公开(公告)日：2024-04-02

申请号：US17470441

申请日：2021-09-09

Applicant: Adobe Inc.

Inventor： Bryan Russell ,  Ruppesh Nalwaya ,  Markus Woodson ,  Joon-Young Lee ,  Hailin Jin

IPC: H04N21/81 ,  G06N3/08 ,  G06V20/40 ,  H04N21/845

CPC classification number: H04N21/8133 ,  G06N3/08 ,  G06V20/46 ,  H04N21/8456

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for automatic tagging of videos. In particular, in one or more embodiments, the disclosed systems generate a set of tagged feature vectors (e.g., tagged feature vectors based on action-rich digital videos) to utilize to generate tags for an input digital video. For instance, the disclosed systems can extract a set of frames for the input digital video and generate feature vectors from the set of frames. In some embodiments, the disclosed systems generate aggregated feature vectors from the feature vectors. Furthermore, the disclosed systems can utilize the feature vectors (or aggregated feature vectors) to identify similar tagged feature vectors from the set of tagged feature vectors. Additionally, the disclosed systems can generate a set of tags for the input digital videos by aggregating one or more tags corresponding to identified similar tagged feature vectors.

公开(公告)号：US11836932B2

公开(公告)日：2023-12-05

申请号：US17350129

申请日：2021-06-17

Applicant: Adobe Inc.

Inventor： Xingyu Liu ,  Hailin Jin ,  Joonyoung Lee

IPC: G06T7/246 ,  G06V20/40 ,  G06V30/19 ,  G06V10/82 ,  G06V10/44 ,  G06F18/2413

CPC classification number: G06T7/246 ,  G06F18/24147 ,  G06V10/454 ,  G06V10/82 ,  G06V20/41 ,  G06V20/46 ,  G06V30/19173 ,  G06T2207/20084

Abstract: Technology is disclosed herein for learning motion in video. In an implementation, an artificial neural network extracts features from a video. A correspondence proposal (CP) module performs, for at least some of the features, a search for corresponding features in the video based on a semantic similarity of a given feature to others of the features. The CP module then generates a joint semantic vector for each of the features based at least on the semantic similarity of the given feature to one or more of the corresponding features and a spatiotemporal distance of the given feature to the one or more of the corresponding features. The artificial neural network is able to identify motion in the video using the joint semantic vectors generated for the features extracted from the video.

公开(公告)号：US11709885B2

公开(公告)日：2023-07-25

申请号：US17025041

申请日：2020-09-18

Applicant: Adobe Inc.

Inventor： John Collomosse ,  Zhe Lin ,  Saeid Motiian ,  Hailin Jin ,  Baldo Faieta ,  Alex Filipkowski

IPC: G06T7/00 ,  G06F16/583 ,  G06F16/532 ,  G06N3/08 ,  G06F16/535 ,  G06V10/82 ,  G06V20/30

CPC classification number: G06F16/5854 ,  G06F16/532 ,  G06F16/535 ,  G06F16/5838 ,  G06N3/08 ,  G06V10/82 ,  G06V20/30

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly identifying digital images with similar style to a query digital image using fine-grain style determination via weakly supervised style extraction neural networks. For example, the disclosed systems can extract a style embedding from a query digital image using a style extraction neural network such as a novel two-branch autoencoder architecture or a weakly supervised discriminative neural network. The disclosed systems can generate a combined style embedding by combining complementary style embeddings from different style extraction neural networks. Moreover, the disclosed systems can search a repository of digital images to identify digital images with similar style to the query digital image. The disclosed systems can also learn parameters for one or more style extraction neural network through weakly supervised training without a specifically labeled style ontology for sample digital images.

公开(公告)号：US20230110114A1

公开(公告)日：2023-04-13

申请号：US17499611

申请日：2021-10-12

Applicant: Adobe Inc.

Inventor： Chinthala Pradyumna Reddy ,  Zhifei Zhang ,  Matthew Fisher ,  Hailin Jin ,  Zhaowen Wang ,  Niloy J Mitra

IPC: G06T11/20 ,  G06T3/40

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media for accurately and flexibly generating scalable fonts utilizing multi-implicit neural font representations. For instance, the disclosed systems combine deep learning with differentiable rasterization to generate a multi-implicit neural font representation of a glyph. For example, the disclosed systems utilize an implicit differentiable font neural network to determine a font style code for an input glyph as well as distance values for locations of the glyph to be rendered based on a glyph label and the font style code. Further, the disclosed systems rasterize the distance values utilizing a differentiable rasterization model and combines the rasterized distance values to generate a permutation-invariant version of the glyph corresponding glyph set.

公开(公告)号：US20210311936A1

公开(公告)日：2021-10-07

申请号：US17350127

申请日：2021-06-17

Applicant: Adobe Inc.

Inventor： Hailin Jin ,  John Collomosse

IPC: G06F16/248 ,  G06N3/08 ,  G06F16/51 ,  G06F16/28 ,  G06F16/532

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for guided visual search. A visual search query can be represented as a sketch sequence that includes ordering information of the constituent strokes in the sketch. The visual search query can be encoded into a structural search encoding in a common search space by a structural neural network. Indexed visual search results can be identified in the common search space and clustered in an auxiliary semantic space. Sketch suggestions can be identified from a plurality of indexed sketches in the common search space. A sketch suggestion can be identified for each semantic cluster of visual search results and presented with the cluster to guide a user towards relevant content through an iterative search process. Selecting a sketch suggestion as a target sketch can automatically transform the visual search query to the target sketch via adversarial images.

公开(公告)号：US11068493B2

公开(公告)日：2021-07-20

申请号：US16183228

申请日：2018-11-07

Applicant: ADOBE INC.

Inventor： Hailin Jin ,  John Collomosse

IPC: G06F16/248 ,  G06N3/08 ,  G06F16/51 ,  G06F16/28 ,  G06F16/532

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for guided visual search. A visual search query can be represented as a sketch sequence that includes ordering information of the constituent strokes in the sketch. The visual search query can be encoded into a structural search encoding in a common search space by a structural neural network. Indexed visual search results can be identified in the common search space and clustered in an auxiliary semantic space. Sketch suggestions can be identified from a plurality of indexed sketches in the common search space. A sketch suggestion can be identified for each semantic cluster of visual search results and presented with the cluster to guide a user towards relevant content through an iterative search process. Selecting a sketch suggestion as a target sketch can automatically transform the visual search query to the target sketch via adversarial images.

公开(公告)号：US20210103783A1

公开(公告)日：2021-04-08

申请号：US17101778

申请日：2020-11-23

Applicant: Adobe Inc.

Inventor： Zhaowen Wang ,  Tianlang Chen ,  Ning Xu ,  Hailin Jin

IPC: G06K9/68 ,  G06F16/906 ,  G06F16/903 ,  G06F40/109 ,  G06N3/08 ,  G06K9/62 ,  G06K9/46

Abstract: The present disclosure relates to a tag-based font recognition system that utilizes a multi-learning framework to develop and improve tag-based font recognition using deep learning neural networks. In particular, the tag-based font recognition system jointly trains a font tag recognition neural network with an implicit font classification attention model to generate font tag probability vectors that are enhanced by implicit font classification information. Indeed, the font recognition system weights the hidden layers of the font tag recognition neural network with implicit font information to improve the accuracy and predictability of the font tag recognition neural network, which results in improved retrieval of fonts in response to a font tag query. Accordingly, using the enhanced tag probability vectors, the tag-based font recognition system can accurately identify and recommend one or more fonts in response to a font tag query.

公开(公告)号：US10963759B2

公开(公告)日：2021-03-30

申请号：US16417115

申请日：2019-05-20

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Mai Long ,  Jonathan Brandt ,  Hailin Jin ,  Chen Fang

IPC: G06K9/66 ,  G06F16/532 ,  G06K9/46 ,  G06K9/62 ,  G06K9/72 ,  G06N3/04 ,  G06F16/583 ,  G06K9/52 ,  G06N3/08

Abstract: The present disclosure includes methods and systems for searching for digital visual media based on semantic and spatial information. In particular, one or more embodiments of the disclosed systems and methods identify digital visual media displaying targeted visual content in a targeted region based on a query term and a query area provide via a digital canvas. Specifically, the disclosed systems and methods can receive user input of a query term and a query area and provide the query term and query area to a query neural network to generate a query feature set. Moreover, the disclosed systems and methods can compare the query feature set to digital visual media feature sets. Further, based on the comparison, the disclosed systems and methods can identify digital visual media portraying targeted visual content corresponding to the query term within a targeted region corresponding to the query area.

公开(公告)号：US20200336802A1

公开(公告)日：2020-10-22

申请号：US16386031

申请日：2019-04-16

Applicant: Adobe Inc.

Inventor： Bryan Russell ,  Ruppesh Nalwaya ,  Markus Woodson ,  Joon-Young Lee ,  Hailin Jin

IPC: H04N21/81 ,  G06K9/00 ,  G06N3/08 ,  H04N21/845

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for automatic tagging of videos. In particular, in one or more embodiments, the disclosed systems generate a set of tagged feature vectors (e.g., tagged feature vectors based on action-rich digital videos) to utilize to generate tags for an input digital video. For instance, the disclosed systems can extract a set of frames for the input digital video and generate feature vectors from the set of frames. In some embodiments, the disclosed systems generate aggregated feature vectors from the feature vectors. Furthermore, the disclosed systems can utilize the feature vectors (or aggregated feature vectors) to identify similar tagged feature vectors from the set of tagged feature vectors. Additionally, the disclosed systems can generate a set of tags for the input digital videos by aggregating one or more tags corresponding to identified similar tagged feature vectors.

公开(公告)号：US10803231B1

公开(公告)日：2020-10-13

申请号：US16369893

申请日：2019-03-29

Applicant: Adobe Inc.

Inventor： Zhaowen Wang ,  Tianlang Chen ,  Ning Xu ,  Hailin Jin

IPC: G06F17/00 ,  G06F40/109 ,  G06K9/68 ,  G06F16/33 ,  G06N3/08 ,  G06K9/62

Abstract: The present disclosure describes a font retrieval system that utilizes a multi-learning framework to develop and improve tag-based font recognition using deep learning neural networks. In particular, the font retrieval system jointly utilizes a combined recognition/retrieval model to generate font affinity scores corresponding to a list of font tags. Further, based on the font affinity scores, the font retrieval system identifies one or more fonts to recommend in response to the list of font tags such that the one or more provided fonts fairly reflect each of the font tags. Indeed, the font retrieval system utilizes a trained font retrieval neural network to efficiently and accurately identify and retrieve fonts in response to a text font tag query.