公开(公告)号：US20110267544A1

公开(公告)日：2011-11-03

申请号：US12768769

申请日：2010-04-28

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Lin-Xie Tang

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Lin-Xie Tang

IPC分类号： H04N5/14

CPC分类号： H04N5/144 ,  G06K9/00751 ,  G11B27/034 ,  G11B27/105 ,  G11B27/28 ,  H04N19/159 ,  H04N19/25 ,  H04N19/61 ,  H04N21/816 ,  H04N21/84 ,  H04N21/8453 ,  H04N21/8455 ,  H04N21/8456 ,  H04N21/8549

摘要： Described is perceptually near-lossless video summarization for use in maintaining video summaries, which operates to substantially reconstruct an original video in a generally perceptually near-lossless manner. A video stream is summarized with little information loss by using a relatively very small piece of summary metadata. The summary metadata comprises an image set of synthesized mosaics and representative keyframes, audio data, and the metadata about video structure and motion. In one implementation, the metadata is computed and maintained (e.g., as a file) to summarize a relatively large video sequence, by segmenting a video shot into subshots, and selecting keyframes and mosaics based upon motion data corresponding to those subshots. The motion data is maintained as a semantic description associated with the image set. To reconstruct the video, the metadata is processed, including simulating motion using the image set and the semantic description, which recovers the audiovisual content without any significant information loss.

摘要翻译： 描述的是用于维护视频摘要的感知上的近无损视频摘要，其操作以基本上以感知方式近无损的方式基本上重建原始视频。 通过使用相对非常小的汇总元数据，视频流总结了很少的信息丢失。 摘要元数据包括合成马赛克的图像集和代表性的关键帧，音频数据以及关于视频结构和运动的元数据。 在一个实施方式中，通过将视频拍摄分割为子照片，并且基于与这些子图片相对应的运动数据来选择关键帧和马赛克，计算和维护元数据（例如，作为文件）来总结相对较大的视频序列。 运动数据被保持为与图像集相关联的语义描述。 为了重建视频，处理元数据，包括使用图像集和语义描述来模拟运动，其恢复视听内容，而没有任何显着的信息丢失。

公开(公告)号：US09628673B2

公开(公告)日：2017-04-18

申请号：US12768769

申请日：2010-04-28

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Lin-Xie Tang

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Lin-Xie Tang

IPC分类号： H04N5/14 ,  G06K9/00 ,  G11B27/034 ,  G11B27/10 ,  G11B27/28 ,  H04N21/81 ,  H04N21/84 ,  H04N21/845 ,  H04N21/8549 ,  H04N19/159 ,  H04N19/61 ,  H04N19/25

CPC分类号： H04N5/144 ,  G06K9/00751 ,  G11B27/034 ,  G11B27/105 ,  G11B27/28 ,  H04N19/159 ,  H04N19/25 ,  H04N19/61 ,  H04N21/816 ,  H04N21/84 ,  H04N21/8453 ,  H04N21/8455 ,  H04N21/8456 ,  H04N21/8549

摘要： Described is perceptually near-lossless video summarization for use in maintaining video summaries, which operates to substantially reconstruct an original video in a generally perceptually near-lossless manner. A video stream is summarized with little information loss by using a relatively very small piece of summary metadata. The summary metadata comprises an image set of synthesized mosaics and representative keyframes, audio data, and the metadata about video structure and motion. In one implementation, the metadata is computed and maintained (e.g., as a file) to summarize a relatively large video sequence, by segmenting a video shot into subshots, and selecting keyframes and mosaics based upon motion data corresponding to those subshots. The motion data is maintained as a semantic description associated with the image set. To reconstruct the video, the metadata is processed, including simulating motion using the image set and the semantic description, which recovers the audiovisual content without any significant information loss.

公开(公告)号：US08804005B2

公开(公告)日：2014-08-12

申请号：US12111202

申请日：2008-04-29

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Zhiwei Gu

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Zhiwei Gu

IPC分类号： G06K9/62 ,  G06K9/34

CPC分类号： G11B27/28 ,  G06K9/00711 ,  G06K9/6282

摘要： Visual concepts contained within a video clip are classified based upon a set of target concepts. The clip is segmented into shots and a multi-layer multi-instance (MLMI) structured metadata representation of each shot is constructed. A set of pre-generated trained models of the target concepts is validated using a set of training shots. An MLMI kernel is recursively generated which models the MLMI structured metadata representation of each shot by comparing prescribed pairs of shots. The MLMI kernel is subsequently utilized to generate a learned objective decision function which learns a classifier for determining if a particular shot (that is not in the set of training shots) contains instances of the target concepts. A regularization framework can also be utilized in conjunction with the MLMI kernel to generate modified learned objective decision functions. The regularization framework introduces explicit constraints which serve to maximize the precision of the classifier.

摘要翻译： 视频剪辑中包含的视觉概念基于一组目标概念进行分类。 剪辑被分割成镜头，并且构建每个镜头的多层多实例（MLMI）结构化元数据表示。 使用一组训练镜头验证了一组预先生成的目标概念训练模型。 通过比较规定的拍摄对，递归地生成MLMI内核，以对每个镜头的MLMI结构化元数据表示进行建模。 MLMI内核随后被用于生成学习的客观决策函数，该函数学习用于确定特定镜头（不在该组训练镜头中）是否包含目标概念的实例的分类器。 正则化框架也可以与MLMI内核一起使用，以生成修改后的学习目标决策函数。 正则化框架引入明确的约束，用于最大化分类器的精度。

公开(公告)号：US08504422B2

公开(公告)日：2013-08-06

申请号：US12786020

申请日：2010-05-24

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Jinlian Guo ,  Fei Sheng

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Jinlian Guo ,  Fei Sheng

IPC分类号： G06Q30/00

CPC分类号： G06F17/30056 ,  G06F17/30047 ,  G06Q30/02 ,  G06Q30/0243

摘要： Techniques for recommending music and advertising to enhance a user's experience while photo browsing are described. In some instances, songs and ads are ranked for relevance to at least one photo from a photo album. The songs, ads and photo(s) from the photo album are then mapped to a style and mood ontology to obtain vector-based representations. The vector-based representations can include real valued terms, each term associated with a human condition defined by the ontology. A re-ranking process generates a relevancy term for each song and each ad indicating relevancy to the photo album. The relevancy terms can be calculated by summing weighted terms from the ranking and the mapping. Recommended music and ads may then be provided to a user, as the user browses a series of photos obtained from the photo album. The ads may be seamlessly embedded into the music in a nonintrusive manner.

摘要翻译： 描述用于推荐音乐和广告以提高用户在照相浏览时体验的技术。 在某些情况下，歌曲和广告的排名与相册中的至少一张照片相关。 然后将相册中的歌曲，广告和照片映射到风格和心境本体以获得基于矢量的表示。 基于向量的表示可以包括实际值，每个术语与由本体定义的人类条件相关联。 重新排序过程产生每个歌曲的相关术语，每个广告指示相册的相关性。 可以通过从排名和映射求和加权项来计算相关项。 然后，当用户浏览从相册获得的一系列照片时，推荐的音乐和广告可以被提供给用户。 广告可以无缝地嵌入到音乐中。

公开(公告)号：US08369686B2

公开(公告)日：2013-02-05

申请号：US12571373

申请日：2009-09-30

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Jinlian Guo

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Jinlian Guo

IPC分类号： H04N9/80

CPC分类号： H04N21/458 ,  G06Q30/02 ,  G06Q30/0243 ,  G11B27/034 ,  G11B27/28 ,  H04N21/44008 ,  H04N21/440236 ,  H04N21/466 ,  H04N21/812

摘要： Video advertising overlay technique embodiments are presented that generally detect a set of spatio-temporal nonintrusive positions within a series of consecutive video frames in shots of a digital video and then overlay contextually relevant ads on these positions. In one general embodiment, this is accomplished by decomposing the video into a series of shots, and then identifying a video advertisement for each of a selected set of the shots. The identified video advertisement is one that is determined to be the most relevant to the content of the shot. An overlay area is also identified in each of the shots, where the selected overlay area is the least intrusive among a plurality of prescribed areas to a viewer of the video. The video advertisements identified for the shots are then respectively scheduled to be overlaid in the identified overlay area of a shot, whenever the shot is played.

摘要翻译： 提供了视频广告覆盖技术实施例，其通常在数字视频的拍摄中检测一系列连续视频帧内的一组时空非侵入位置，然后在这些位置上重叠相关的相关广告。 在一个一般实施例中，这通过将视频分解成一系列镜头，然后为所选择的一组拍摄中的每一个识别视频广告来实现。 所识别的视频广告是被确定为与拍摄内容最相关的广告。 在每个拍摄中还识别覆盖区域，其中所选覆盖区域在多个规定区域中对于视频的观看者是最小的侵入。 每当拍摄被拍摄时，为拍摄而识别的视频广告然后分别被调度为覆盖在所识别的拍摄的重叠区域中。

公开(公告)号：US20110288929A1

公开(公告)日：2011-11-24

申请号：US12786020

申请日：2010-05-24

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Jinlian Guo ,  Fei Sheng

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li ,  Jinlian Guo ,  Fei Sheng

IPC分类号： G06Q30/00 ,  G06F17/30 ,  G06Q10/00

CPC分类号： G06F17/30056 ,  G06F17/30047 ,  G06Q30/02 ,  G06Q30/0243

摘要： Techniques for recommending music and advertising to enhance a user's experience while photo browsing are described. In some instances, songs and ads are ranked for relevance to at least one photo from a photo album. The songs, ads and photo(s) from the photo album are then mapped to a style and mood ontology to obtain vector-based representations. The vector-based representations can include real valued terms, each term associated with a human condition defined by the ontology. A re-ranking process generates a relevancy term for each song and each ad indicating relevancy to the photo album. The relevancy terms can be calculated by summing weighted terms from the ranking and the mapping. Recommended music and ads may then be provided to a user, as the user browses a series of photos obtained from the photo album. The ads may be seamlessly embedded into the music in a nonintrusive manner.

摘要翻译： 描述用于推荐音乐和广告以提高用户在照相浏览时体验的技术。 在某些情况下，歌曲和广告的排名与相册中的至少一张照片相关。 然后将相册中的歌曲，广告和照片映射到风格和心境本体以获得基于矢量的表示。 基于向量的表示可以包括实际值，每个术语与由本体定义的人类条件相关联。 重新排序过程产生每个歌曲的相关术语，每个广告指示相册的相关性。 可以通过从排名和映射求和加权项来计算相关项。 然后，当用户浏览从相册获得的一系列照片时，推荐的音乐和广告可以被提供给用户。 广告可以无缝地嵌入到音乐中。

公开(公告)号：US20090003712A1

公开(公告)日：2009-01-01

申请号：US12055267

申请日：2008-03-25

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li

IPC分类号： G06K9/62

CPC分类号： G06F16/745 ,  G06F16/739 ,  G06K9/00744 ,  G06K9/3233

摘要： A method, a computer-readable storage media, and a user interface describe techniques for creating a video collage synthesized from video content, selecting representative images from the video content, extracting and resizing regions of interest (ROI) from the representative images from the video content, and arranging the regions of interest on a canvas without seams while preserving a temporal structure of the video content. The described method, computer-readable storage, and user interface enhance the experience of the user in browsing a video collage that is compact.

摘要翻译： 一种方法，计算机可读存储介质和用户界面描述了用于创建从视频内容合成的视频拼贴的技术，从视频内容中选择代表图像，从视频中提取和调整来自代表图像的感兴趣区域（ROI） 内容，并且在没有接缝的情况下在画布上排列感兴趣的区域，同时保留视频内容的时间结构。 所描述的方法，计算机可读存储和用户界面增强了用户浏览紧凑的视频拼贴的体验。

公开(公告)号：US20120109754A1

公开(公告)日：2012-05-03

申请号：US12939132

申请日：2010-11-03

申请人： Tao Mei ,  Xian-Sheng Hua ,  Ying-Qing Xu ,  Shipeng Li

发明人： Tao Mei ,  Xian-Sheng Hua ,  Ying-Qing Xu ,  Shipeng Li

IPC分类号： G06Q30/00 ,  G06F15/16

CPC分类号： G06Q30/0251 ,  G06Q30/0267

摘要： The sponsored multi-media blogging technique is an advertising-driven service on a computing device, such as a mobile phone, that makes the multi-media micro-blog or blog an effective carrier for advertising. The data collected while employing the sponsored multi-media blogging technique is used for user intent mining and increasing advertisement relevance for mobile advertising projects. The benefits to the sponsored multi-media blogging technique's users are a natural interface for composing multi-media micro-blogs/blogs and instant experience sharing, while the benefits to advertisers is the promoted brand impression from the contextual advertising in rich media micro-blogs/blogs.

摘要翻译： 赞助的多媒体博客技术是在诸如手机之类的计算设备上的广告驱动服务，使得多媒体微博或博客成为广告的有效载体。 采用赞助的多媒体博客技术收集的数据用于用户意图挖掘和增加移动广告项目的广告相关性。 赞助的多媒体博客技术用户的好处是构成多媒体微博/博客和即时体验共享的自然界面，而对广告客户的好处是从富媒体微博上下文广告中提升的品牌印象 /博客。

公开(公告)号：US20120095825A1

公开(公告)日：2012-04-19

申请号：US12906899

申请日：2010-10-18

申请人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li

发明人： Tao Mei ,  Xian-Sheng Hua ,  Shipeng Li

IPC分类号： G06Q30/00

CPC分类号： G06Q30/0276 ,  G06Q30/0244 ,  G06Q30/0245

摘要： Techniques for image selection and region of interest analysis are described herein. A pair of two or more users is configured, and an image is displayed to the pair. The image can be a still image (i.e., a picture) or a moving image (i.e., video). In some instances, a plurality of advertisements is suggested for possible association with the image. Input is received from both users in the pair, indicating a positive or a negative association between each advertisement and the image. When the pair positively rates an advertisement, the advertisement is associated with the image. A plurality of regions of interest within the image may be suggested. In response, positive or negative input is received from the pair indicating whether each of the plurality of regions of interest is appropriately suggested for placement of an advertisement.

摘要翻译： 本文描述了用于图像选择和感兴趣区域分析的技术。 配置一对两个以上的用户，并且一对图像被显示。 图像可以是静止图像（即，图像）或运动图像（即，视频）。 在一些情况下，建议多个广告以与图像可能的关联。 从该对中的两个用户接收到输入，指示每个广告和图像之间的正或负关联。 当该对对广告进行正比率时，该广告与图像相关联。 可以建议图像内的多个感兴趣区域。 作为响应，从该对接收到正或负输入，指示是否适当地建议多个感兴趣区域中的每一个用于放置广告。

公开(公告)号：US07773813B2

公开(公告)日：2010-08-10

申请号：US11263081

申请日：2005-10-31

申请人： Xian-Sheng Hua ,  Shipeng Li ,  Tao Mei

发明人： Xian-Sheng Hua ,  Shipeng Li ,  Tao Mei

IPC分类号： G06K9/62

CPC分类号： G06F17/30811 ,  G06K9/00711 ,  G11B27/28 ,  H04N21/4147 ,  H04N21/4223 ,  H04N21/4334 ,  H04N21/44008 ,  H04N21/84 ,  H04N21/8456 ,  H04N21/854

摘要： Systems and methods are described for detecting capture-intention in order to analyze video content. In one implementation, a system decomposes video structure into sub-shots, extracts intention-oriented features from the sub-shots, delineates intention units via the extracted features, and classifies the intention units into intention categories via the extracted features. A video library can be organized via the categorized intention units.

摘要翻译： 描述了用于检测捕获意图以分析视频内容的系统和方法。 在一个实现中，系统将视频结构分解为子照片，从子拍摄中提取面向意图的特征，通过提取的特征描绘意图单元，并通过提取的特征将意图单元分类成意图类别。 可以通过分类意图单位组织视频库。