公开(公告)号：US20180267996A1

公开(公告)日：2018-09-20

申请号：US15463757

申请日：2017-03-20

Applicant: ADOBE SYSTEMS INCORPORATED

Inventor： ZHE LIN ,  XIAOHUI SHEN ,  JIANMING ZHANG ,  HAILIN JIN ,  YINGWEI LI

IPC: G06F17/30 ,  G06T11/60 ,  G06T7/33 ,  G06K9/66 ,  G06N3/04 ,  G06N3/08

CPC classification number: G06F16/5866 ,  G06F16/532 ,  G06F16/951 ,  G06K9/00684 ,  G06K9/4628 ,  G06K9/4676 ,  G06K9/6248 ,  G06K9/6273 ,  G06K9/66 ,  G06N3/04 ,  G06N3/0454 ,  G06N3/08 ,  G06T7/33 ,  G06T11/60 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A framework is provided for associating dense images with topics. The framework is trained utilizing images, each having multiple regions, multiple visual characteristics and multiple keyword tags associated therewith. For each region of each image, visual features are computed from the visual characteristics utilizing a convolutional neural network, and an image feature vector is generated from the visual features. The keyword tags are utilized to generate a weighted word vector for each image by calculating a weighted average of word vector representations representing keyword tags associated with the image. The image feature vector and the weighted word vector are aligned in a common embedding space and a heat map is computed for the image. Once trained, the framework can be utilized to automatically tag images and rank the relevance of images with respect to queried keywords based upon associated heat maps.

公开(公告)号：US20170236055A1

公开(公告)日：2017-08-17

申请号：US15094633

申请日：2016-04-08

Applicant: ADOBE SYSTEMS INCORPORATED

Inventor： ZHE LIN ,  XIAOHUI SHEN ,  JONATHAN BRANDT ,  JIANMING ZHANG ,  CHEN FANG

IPC: G06N3/08 ,  G06N3/04

CPC classification number: G06N3/08 ,  G06F17/30247 ,  G06N3/0454 ,  G06N3/0472 ,  G06N99/005

Abstract: Embodiments of the present invention provide an automated image tagging system that can predict a set of tags, along with relevance scores, that can be used for keyword-based image retrieval, image tag proposal, and image tag auto-completion based on user input. Initially, during training, a clustering technique is utilized to reduce cluster imbalance in the data that is input into a convolutional neural network (CNN) for training feature data. In embodiments, the clustering technique can also be utilized to compute data point similarity that can be utilized for tag propagation (to tag untagged images). During testing, a diversity based voting framework is utilized to overcome user tagging biases. In some embodiments, bigram re-weighting can down-weight a keyword that is likely to be part of a bigram based on a predicted tag set.

公开(公告)号：US20180267997A1

公开(公告)日：2018-09-20

申请号：US15463769

申请日：2017-03-20

Applicant: ADOBE SYSTEMS INCORPORATED

Inventor： ZHE LIN ,  XIAOHUI SHEN ,  JIANMING ZHANG ,  HAILIN JIN ,  YINGWEI LI

IPC: G06F17/30 ,  G06T11/60 ,  G06T7/33 ,  G06K9/66 ,  G06N3/04 ,  G06N3/08

CPC classification number: G06F17/30268 ,  G06F17/30277 ,  G06F17/30864 ,  G06K9/00684 ,  G06K9/66 ,  G06N3/04 ,  G06N3/08 ,  G06T7/33 ,  G06T11/60 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A framework is provided for associating images with topics utilizing embedding learning. The framework is trained utilizing images, each having multiple visual characteristics and multiple keyword tags associated therewith. Visual features are computed from the visual characteristics utilizing a convolutional neural network and an image feature vector is generated therefrom. The keyword tags are utilized to generate a weighted word vector (or “soft topic feature vector”) for each image by calculating a weighted average of word vector representations that represent the keyword tags associated with the image. The image feature vector and the soft topic feature vector are aligned in a common embedding space and a relevancy score is computed for each of the keyword tags. Once trained, the framework can automatically tag images and a text-based search engine can rank image relevance with respect to queried keywords based upon predicted relevancy scores.

公开(公告)号：US20170236032A1

公开(公告)日：2017-08-17

申请号：US15043174

申请日：2016-02-12

Applicant: ADOBE SYSTEMS INCORPORATED

Inventor： ZHE LIN ,  XIAOHUI SHEN ,  JONATHAN BRANDT ,  JIANMING ZHANG ,  CHEN FANG

IPC: G06K9/62 ,  G06N3/08 ,  G06F17/30

CPC classification number: G06K9/623 ,  G06F16/24578 ,  G06F16/285 ,  G06F16/51 ,  G06F16/583 ,  G06K9/4628 ,  G06K9/6223 ,  G06K9/6262 ,  G06K9/6276 ,  G06N3/0454 ,  G06N3/08 ,  G06N20/10

Abstract: Embodiments of the present invention provide an automated image tagging system that can predict a set of tags, along with relevance scores, that can be used for keyword-based image retrieval, image tag proposal, and image tag auto-completion based on user input. Initially, during training, a clustering technique is utilized to reduce cluster imbalance in the data that is input into a convolutional neural network (CNN) for training feature data. In embodiments, the clustering technique can also be utilized to compute data point similarity that can be utilized for tag propagation (to tag untagged images). During testing, a diversity based voting framework is utilized to overcome user tagging biases. In some embodiments, bigram re-weighting can down-weight a keyword that is likely to be part of a bigram based on a predicted tag set.