公开(公告)号：US10026020B2

公开(公告)日：2018-07-17

申请号：US14997011

申请日：2016-01-15

Applicant: Adobe Systems Incorporated

Inventor： Hailin Jin ,  Zhou Ren ,  Zhe Lin ,  Chen Fang

IPC: G06K9/62 ,  G06F17/30 ,  G06K9/66 ,  G06K9/72 ,  G06K9/00

CPC classification number: G06K9/628 ,  G06F16/5846 ,  G06F16/5866 ,  G06K9/00684 ,  G06K9/66 ,  G06K9/726

Abstract: Embedding space for images with multiple text labels is described. In the embedding space both text labels and image regions are embedded. The text labels embedded describe semantic concepts that can be exhibited in image content. The embedding space is trained to semantically relate the embedded text labels so that labels like “sun” and “sunset” are more closely related than “sun” and “bird”. Training the embedding space also includes mapping representative images, having image content which exemplifies the semantic concepts, to respective text labels. Unlike conventional techniques that embed an entire training image into the embedding space for each text label associated with the training image, the techniques described herein process a training image to generate regions that correspond to the multiple text labels. The regions of the training image are then embedded into the training space in a manner that maps the regions to the corresponding text labels.

公开(公告)号：US09811765B2

公开(公告)日：2017-11-07

申请号：US14995032

申请日：2016-01-13

Applicant: Adobe Systems Incorporated

Inventor： Zhaowen Wang ,  Quanzeng You ,  Hailin Jin ,  Chen Fang

IPC: G06K9/68 ,  G06K9/62 ,  G06K9/46 ,  G06F17/30 ,  G06N3/08 ,  G06N3/04 ,  G06N7/00

CPC classification number: G06K9/6269 ,  G06F17/30247 ,  G06F17/3028 ,  G06F17/30675 ,  G06K9/00664 ,  G06K9/4604 ,  G06K9/4628 ,  G06K9/6202 ,  G06K9/6274 ,  G06N3/0445 ,  G06N3/08 ,  G06N7/005

Abstract: Techniques for image captioning with weak supervision are described herein. In implementations, weak supervision data regarding a target image is obtained and utilized to provide detail information that supplements global image concepts derived for image captioning. Weak supervision data refers to noisy data that is not closely curated and may include errors. Given a target image, weak supervision data for visually similar images may be collected from sources of weakly annotated images, such as online social networks. Generally, images posted online include “weak” annotations in the form of tags, titles, labels, and short descriptions added by users. Weak supervision data for the target image is generated by extracting keywords for visually similar images discovered in the different sources. The keywords included in the weak supervision data are then employed to modulate weights applied for probabilistic classifications during image captioning analysis.

公开(公告)号：US20160379132A1

公开(公告)日：2016-12-29

申请号：US14748059

申请日：2015-06-23

Applicant: Adobe Systems Incorporated

Inventor： Hailin Jin ,  Chen Fang ,  Jianchao Yang ,  Zhe Lin

IPC: G06N99/00 ,  H04L12/26 ,  H04L29/08

Abstract: The present disclosure is directed to collaborative feature learning using social media data. For example, a machine learning system may identify social media data that includes user behavioral data, which indicates user interactions with content item. Using the identified social user behavioral data, the machine learning system may determine latent representations from the content items. In some embodiments, the machine learning system may train a machine-learning model based on the latent representations. Further, the machine learning system may extract features of the content item from the trained machine-learning model.

Abstract translation: 本公开涉及使用社交媒体数据的协作特征学习。 例如，机器学习系统可以识别包括用户行为数据的社交媒体数据，其指示用户与内容项目的交互。 使用所识别的社会用户行为数据，机器学习系统可以确定来自内容项目的潜在表示。 在一些实施例中，机器学习系统可以基于潜在表示来训练机器学习模型。 此外，机器学习系统可以从训练的机器学习模型中提取内容项的特征。

公开(公告)号：US20150213609A1

公开(公告)日：2015-07-30

申请号：US14169073

申请日：2014-01-30

Applicant: Adobe Systems Incorporated

Inventor： Zhe Lin ,  Radomir Mech ,  Xiaohui Shen ,  Chen Fang

IPC: G06T7/00

CPC classification number: G06T7/0081 ,  G06F3/04845 ,  G06T7/00 ,  G06T11/60 ,  G06T2207/10004 ,  G06T2207/20016 ,  G06T2207/20081 ,  G06T2207/20132 ,  G06T2207/30168

Abstract: Image cropping suggestion is described. In one or more implementations, multiple croppings of a scene are scored based on parameters that indicate visual characteristics established for visually pleasing croppings. The parameters may include a parameter that indicates composition quality of a candidate cropping, for example. The parameters may also include a parameter that indicates whether content appearing in the scene is preserved and a parameter that indicates simplicity of a boundary of a candidate cropping. Based on the scores, image croppings may be chosen, e.g., to present the chosen image croppings to a user for selection. To choose the croppings, they may be ranked according to the score and chosen such that consecutively ranked croppings are not chosen. Alternately or in addition, image croppings may be chosen that are visually different according to scores which indicate those croppings have different visual characteristics.

Abstract translation: 描述了图像裁剪建议。 在一个或多个实现中，基于指示为视觉上令人满意的裁剪而建立的视觉特征的参数对场景进行多次裁剪。 参数可以包括例如表示候选裁剪的组合质量的参数。 参数还可以包括指示是否保存出现在场景中的内容的参数以及指示候选裁剪边界的简单性的参数。 基于分数，可以选择图像裁切，例如，将所选择的图像裁切呈现给用户进行选择。 要选择裁剪，可以根据分数进行排序，并选择不选择连续排序的裁剪。 或者或另外，可以根据指示这些裁剪具有不同视觉特征的分数在视觉上不同地选择图像裁切。