公开(公告)号：US07599554B2

公开(公告)日：2009-10-06

申请号：US10552829

申请日：2004-04-02

申请人： Lalitha Agnihotri ,  Nevenka Dimitrova ,  John Kender

发明人： Lalitha Agnihotri ,  Nevenka Dimitrova ,  John Kender

IPC分类号： G06K9/34

CPC分类号： G06K9/00711 ,  G06F17/30793 ,  G06F17/30796 ,  G06F17/30843 ,  H04H60/58

摘要： A method and apparatus are provided for segmenting and summarizing a music video (507) in a multimedia stream (505) using content analysis. A music video (507) is segmented in a multimedia stream (505) by evaluating a plurality of content features that are related to the multimedia stream. The plurality of content features includes at least two of a face presence feature; a videotext presence feature; a color histogram feature; an audio feature, a camera cut feature; and an analysis of key words obtained from a transcript of the at least one music video. The plurality of content features are processed using a pattern recognition engine (1000), such as a Bayesian Belief Network, or one or more video segmentation rules (1115) to identify the music video (507) in the multimedia stream (505). A chorus is detected in at least one music video (507) using a transcript (T) of the music video (507) based upon a repetition of words in the transcript. The extracted chorus may be employed for the automatic generation of a summary of the music video (507).

摘要翻译： 提供了一种使用内容分析在多媒体流（505）中分割和汇总音乐视频（507）的方法和装置。 通过评估与多媒体流相关的多个内容特征，音乐视频（507）在多媒体流（505）中被分割。 多个内容特征包括面部存在特征中的至少两个; 录像带存在功能; 颜色直方图特征; 音频功能，相机切割功能; 以及从至少一个音乐视频的抄本获得的关键词的分析。 使用诸如贝叶斯信仰网络的模式识别引擎（1000）或者识别多媒体流（505）中的音乐视频（507）的一个或多个视频分段规则（1115）来处理多个内容特征。 基于抄本中的单词的重复，使用音乐视频（507）的抄本（T）在至少一个音乐视频（507）中检测到合唱。 提取的合唱可以用于自动生成音乐视频的摘要（507）。

公开(公告)号：US20080187231A1

公开(公告)日：2008-08-07

申请号：US11817798

申请日：2006-03-03

申请人： Mauro Barbieri ,  Nevenka Dimitrova ,  Lalitha Agnihotri

发明人： Mauro Barbieri ,  Nevenka Dimitrova ,  Lalitha Agnihotri

IPC分类号： G06K9/46

CPC分类号： G06K9/00718 ,  G06F16/739 ,  G06F16/784 ,  G06F16/7844 ,  G06K9/00751

摘要： Summarization of audio and/or visual data based on clustering of object type features is disclosed. Summaries of video, audio and/or audiovisual data may be provided without any need of knowledge about the true identity of the objects that are present in the data. In one embodiment of the invention are video summaries of movies provided. The summarization comprising the steps of inputting audio and/or visual data, locating an object in a frame of the data, such as locating a face of an actor, extracting type features of the located object in the frame. The extraction of type features is done for a plurality of frames and similar type features are grouped together in individual clusters, each cluster being linked to an identity of the object. After the processing of the video content, the largest clusters correspond to the most important persons in the video.

摘要翻译： 公开了基于对象类型特征聚类的音频和/或视觉数据的总结。 可以提供视频，音频和/或视听数据的摘要，而不需要知道数据中存在的对象的真实身份。 在本发明的一个实施例中，提供了电影的视频摘要。 总结包括以下步骤：输入音频和/或视觉数据，将对象定位在数据的帧中，例如定位演员的脸部，提取帧中的定位对象的类型特征。 对多个帧进行类型特征的提取，并且将类似的类型特征分组在各个簇中，每个簇与对象的身份相关联。 在处理视频内容之后，最大的集群对应于视频中最重要的人物。

公开(公告)号：US07373336B2

公开(公告)日：2008-05-13

申请号：US10165904

申请日：2002-06-10

申请人： Radu S. Jasinschi ,  Nevenka Dimitrova ,  John Zimmerman

发明人： Radu S. Jasinschi ,  Nevenka Dimitrova ,  John Zimmerman

IPC分类号： G06F17/30

CPC分类号： H04N21/475 ,  G06F17/30828 ,  H04N7/17318 ,  H04N21/26603 ,  H04N21/44222 ,  H04N21/4532 ,  H04N21/454 ,  H04N21/4622 ,  H04N21/4722 ,  H04N21/4782 ,  H04N21/84 ,  H04N21/858 ,  Y10S707/99933 ,  Y10S707/99934 ,  Y10S707/99935

摘要： A method, process and system for performing content augmentation of personal profiles includes (a) building a user history of a plurality of augmented content information of relevant TV programs; (b) analyzing user queries and determining a degree to which the user queried for additional content information; (c) inferring values about the user from user queries for additional content information so as to augment the additional content information; (d) updating the augmented content information to at least one of the user history, Internet and specialized databases; (e) linking individual ones of the plurality of augmented content information to each other; and (f) determining inferences about the user's interests and preferences based on the linkage of the plurality of augmented content information. The updating of the augmented content information includes segmenting and indexing of multimedia content. A feedback system is created where user queries for more information and purchases from the Internet and specialized databases will result in additional augmented content information about the particular user.

摘要翻译： 用于执行个人简档的内容增加的方法，过程和系统包括（a）构建相关电视节目的多个增强内容信息的用户历史; （b）分析用户查询并确定用户查询附加内容信息的程度; （c）从附加内容信息的用户查询推断关于用户的值，以便增加附加内容信息; （d）将增强内容信息更新为用户历史，因特网和专用数据库中的至少一个; （e）将所述多个增强内容信息中的各个彼此连接; 以及（f）基于所述多个增强内容信息的链接来确定关于所述用户的兴趣和偏好的推断。 增强内容信息的更新包括多媒体内容的分段和索引。 创建反馈系统，其中用户从因特网和专用数据库查询更多信息和购买将导致关于特定用户的附加增强内容信息。

公开(公告)号：US07170566B2

公开(公告)日：2007-01-30

申请号：US10028378

申请日：2001-12-21

申请人： Thomas McGee ,  Lalitha Agnihotri ,  Nevenka Dimitrova ,  Radu Serban Jasinschi

发明人： Thomas McGee ,  Lalitha Agnihotri ,  Nevenka Dimitrova ,  Radu Serban Jasinschi

IPC分类号： H04N5/222

CPC分类号： H04N5/44 ,  H04N21/44008 ,  H04N21/812 ,  H04N21/84 ,  H04N21/8456

摘要： Techniques are disclosed for detecting commercials or other particular types of video content in a video signal. In an illustrative embodiment, color histograms are extracted from frames of the video signal. For each of at least a subset of the extracted color histograms, the extracted color histogram is compared to a family histogram. If the extracted color histogram falls within a specified range of the family histogram, the family histogram is updated to include the extracted color histogram as a new member. If the extracted color histogram does not fall within the specified range of the family histogram, the family histogram is considered complete and the extracted color histogram is utilized to generate a new family histogram for use in processing subsequent extracted color histograms. The resulting family histograms are utilized to detect commercials or other particular type of video content in the video signal.

摘要翻译： 公开了用于检测视频信号中的商业广告或其他特定类型的视频内容的技术。 在说明性实施例中，从视频信号的帧中提取颜色直方图。 对于提取的颜色直方图的至少一个子集中的每一个，将所提取的颜色直方图与族直方图进行比较。 如果提取的颜色直方图落入家族直方图的指定范围内，则更新系列直方图以将所提取的颜色直方图包括为新成员。 如果提取的颜色直方图不在家庭直方图的指定范围内，则家族直方图被认为是完整的，并且所提取的颜色直方图被用于生成用于处理随后提取的颜色直方图的新家族直方图。 所得到的系列直方图用于检测视频信号中的广告或其他特定类型的视频内容。

公开(公告)号：US07120626B2

公开(公告)日：2006-10-10

申请号：US10295668

申请日：2002-11-15

申请人： Dongge Li ,  Nevenka Dimitrova

发明人： Dongge Li ,  Nevenka Dimitrova

IPC分类号： G06F17/30

CPC分类号： G06K9/00711 ,  G06F17/30026 ,  G06F17/30035 ,  G06F17/30047 ,  G06F17/30743 ,  G06F17/30775 ,  G06F17/30787 ,  G06F17/30799 ,  Y10S707/99933

摘要： A method and system which enable a user to query a multimedia archive in one media modality and automatically retrieve correlating data in another media modality without the need for manually associating the data items through a data structure. The correlation method finds the maximum correlation between the data items without being affected by the distribution of the data in the respective subspace of each modality. Once the direction of correlation is disclosed, extracted features can be transferred from one subspace to another.

摘要翻译： 一种方法和系统，其使得用户能够以一种媒体模式查询多媒体档案并且自动地检索另一种媒体模式中的相关数据，而不需要通过数据结构手动关联数据项。 相关方法在数据项之间找到最大的相关性，而不受每个模态各个子空间中数据分布的影响。 一旦公开了相关方向，提取的特征可以从一个子空间转移到另一个子空间。

公开(公告)号：US20060165379A1

公开(公告)日：2006-07-27

申请号：US10562538

申请日：2004-06-28

申请人： Lalitha Agnihotri ,  Nevenka Dimitrova

发明人： Lalitha Agnihotri ,  Nevenka Dimitrova

IPC分类号： H04N5/91

CPC分类号： H04N21/4532 ,  G06F17/30787 ,  G06F17/30796 ,  G06F17/30802 ,  G06F17/30805 ,  G06F17/30811 ,  G06F17/30828 ,  G06F17/30843 ,  G11B27/28 ,  H04N5/147 ,  H04N5/44543 ,  H04N7/165 ,  H04N7/17318 ,  H04N7/17336 ,  H04N21/2335 ,  H04N21/234354 ,  H04N21/2402 ,  H04N21/25808 ,  H04N21/25891 ,  H04N21/26216 ,  H04N21/2662 ,  H04N21/4755 ,  H04N21/6131 ,  H04N21/6582 ,  H04N21/84 ,  H04N21/8453 ,  H04N21/8549

摘要： A system facilitates and enhances review of one or more multimedia input streams that includes some combination of video, audio and text information, generating a multimedia summary, thereby enabling a user to better browse and/or decide on viewing the multimedia input streams in their entirety. The multimedia summary is constructed automatically, based in part on system specifications, user specifications and network and device constraints. In a particular application of the invention, the input multimedia streams represent news broadcasts (e.g., television news program, video vault footage). In such a particular application, the invention can enable the user to automatically receive a summary of the news stream in accordance with previously provided user preferences and in accordance with prevailing network and user device constraints.

摘要翻译： 系统有助于和增强对包括视频，音频和文本信息的某些组合的一个或多个多媒体输入流的审查，从而产生多媒体摘要，从而使用户能够更好地浏览和/或决定整体观看多媒体输入流 。 部分基于系统规格，用户规格和网络和设备限制，自动构建多媒体摘要。 在本发明的特定应用中，输入多媒体流代表新闻广播（例如电视新闻节目，视频保险库镜头）。 在这样的特定应用中，本发明可以使得用户能够根据先前提供的用户偏好并根据当前的网络和用户设备约束来自动接收新闻流的摘要。

公开(公告)号：US06990496B1

公开(公告)日：2006-01-24

申请号：US09616631

申请日：2000-07-26

申请人： Thomas Francis McGee, III ,  Nevenka Dimitrova

发明人： Thomas Francis McGee, III ,  Nevenka Dimitrova

IPC分类号： G06F17/00

CPC分类号： G06F17/30796 ,  G06F17/30746 ,  G06K9/62 ,  Y10S707/99942 ,  Y10S707/99943 ,  Y10S707/99945

摘要： For use in an information processing system, there is disclosed a system and method for automatically classifying text. The system comprises a text classifier controller that reads text having one or more keywords contained within one or more story segments within the text. The text classifier controller identifies keywords within each line, and, in response to identifying at least one keyword within a line of text, classifies that line of text as a part of a story segment within the text. The text classifier controller also identifies keyword transition points in the text where the number of detected keywords in a particular category of keywords decreases below a threshold number. The text classifier controller also identifies keyword transition points in the text where the number of detected keywords in a particular category of keywords increases above a threshold number. The text classifier controller classifies story segments based on the location of the keyword transition points.

摘要翻译： 为了在信息处理系统中使用，公开了一种用于自动对文本进行分类的系统和方法。 该系统包括文本分类器控制器，其读取具有包含在文本内的一个或多个故事段内的一个或多个关键字的文本。 文本分类器控制器识别每行内的关键字，并且响应于在文本行内识别至少一个关键字，将文本行作为文本中的故事段的一部分进行分类。 文本分类器控制器还识别文本中的关键字转换点，其中特定关键类别的检测到的关键字的数量减少到阈值以下。 文本分类器控制器还识别文本中的关键字转换点，其中特定关键词类别中检测到的关键字的数量增加到阈值数以上。 文本分类器控制器根据关键字转换点的位置对故事段进行分类。

公开(公告)号：US06819863B2

公开(公告)日：2004-11-16

申请号：US09746077

申请日：2000-12-22

申请人： Serhan Dagtas ,  Nevenka Dimitrova

发明人： Serhan Dagtas ,  Nevenka Dimitrova

IPC分类号： H04N591

CPC分类号： G06K9/00711 ,  G06F17/30787 ,  G06F17/30796 ,  G11B27/22 ,  G11B27/28 ,  G11B2220/216 ,  G11B2220/2516 ,  G11B2220/2545 ,  G11B2220/2562 ,  G11B2220/455 ,  G11B2220/90 ,  H04N21/4394 ,  H04N21/812 ,  Y10S358/908

摘要： For use in a video signal processor, there is disclosed a system and method for locating program boundaries and commercial boundaries using audio categories. The system comprises an audio classifier controller that obtains information concerning the audio categories of the segments of an audio signal. Audio categories include such categories as silence, music, noise and speech. The audio classifier controller determines the rates of change of the audio categories. The audio classifier controller then compares each rate of change of the audio categories with a threshold value to locate the boundaries of the programs and commercials. The audio classifier controller is also capable of classifying at least one feature of an audio category change rate using a multifeature classifier to locate the boundaries of the programs and commercials.

摘要翻译： 为了在视频信号处理器中使用，公开了一种用于使用音频类别来定位节目边界和商业边界的系统和方法。 该系统包括音频分类器控制器，其获得关于音频信号的段的音频类别的信息。 音频类别包括静音，音乐，噪音和语音等类别。 音频分类器控制器确定音频类别的变化率。 然后，音频分类器控制器将音频类别的每个变化率与阈值进行比较，以定位节目和广告的边界。 音频分类器控制器还能够使用多重分类器对音频类别变化率的至少一个特征进行分类，以定位节目和广告的边界。

公开(公告)号：US06608930B1

公开(公告)日：2003-08-19

申请号：US09370931

申请日：1999-08-09

申请人： Lalitha Agnihotri ,  Nevenka Dimitrova ,  Jan H. Elenbaas

发明人： Lalitha Agnihotri ,  Nevenka Dimitrova ,  Jan H. Elenbaas

IPC分类号： G06K934

CPC分类号： G06F17/30796 ,  G06F17/30805 ,  G06F17/30817 ,  G06F17/3084 ,  G06K9/3266

摘要： There is disclosed, for use in video text analysis system, a video processing device for searching video streams for one or more user-selected image text attributes. The video processing device comprises an image processor capable detecting and extracting image text from video frames, determining attributes of the extracted image text, comparing the extracted image text attributes and the user-selected image text attributes, and, if a match occurs, modifying, transferring, and/or labeling at least a portion of the video stream in accordance with user commands. The invention uses the user-selected image text attributes to search through an archive of video clips to 1) locate particular types of events, such as news programs or sports events; 2) locate programs featuring particular persons or groups; 3) locate programs by name; 4) save or remove all or some commercials, and to otherwise sort, edit, and save all of, or portions of, video clips according to image text that appears in the frames of the video clips.

摘要翻译： 公开了用于视频文本分析系统中的用于搜索一个或多个用户选择的图像文本属性的视频流的视频处理设备。 视频处理装置包括能够检测和提取来自视频帧的图像文本，确定提取的图像文本的属性，比较提取的图像文本属性和用户选择的图像文本属性的图像处理装置，并且如果匹配发生， 根据用户命令传送和/或标记视频流的至少一部分。 本发明使用用户选择的图像文本属性来搜索视频剪辑的归档以1）定位特定类型的事件，例如新闻节目或体育赛事; 2）查找特定个人或团体的节目; 3）按名称查找程序; 4）保存或删除所有或某些商业广告，并根据出现在视频剪辑的帧中的图像文本，对视频片段的全部或部分进行排序，编辑和保存。

公开(公告)号：US06119123A

公开(公告)日：2000-09-12

申请号：US982972

申请日：1997-12-02

申请人： Jan Hermanus Elenbaas ,  Nevenka Dimitrova

发明人： Jan Hermanus Elenbaas ,  Nevenka Dimitrova

IPC分类号： G06F17/30 ,  H04N5/76 ,  H04N5/91 ,  G06F9/00

CPC分类号： G06F17/30852 ,  G06F17/30858 ,  Y10S707/99943 ,  Y10S707/99945

摘要： A method and apparatus for forming a visual index of scenes in a video image which has been or is being recorded in a computer readable memory. A selected number of keyframes are derived from the recorded image, each being representative of a respective scene therein. The keyframes are then ordered into a selected number of levels of detail of the scenes represented thereby, each level including a predetermined number of keyframes, each subsequent level including keyframes of greater detail than those in a preceding level. A header file is then formed which is descriptive of the ordered set of keyframes, and the header file is stored together with the ordered set of keyframes in the computer readable memory. A user can thereby identify and obtain optimized retrieval in accordance with his preferences of particular segments of the video image from a relatively slow memory device. The method and apparatus are equally applicable to formation of an indexed order of binary large objects ("blobs") in a set of multimedia documents in accordance with a user's preferences.

摘要翻译： 一种用于形成已经或正在记录在计算机可读存储器中的视频图像中的场景的视觉索引的方法和装置。 从所记录的图像中导出所选数量的关键帧，每个都代表其中的相应场景。 然后将关键帧排列成由其表示的场景的所选数量的细节级别，每个级别包括预定数量的关键帧，每个后续级别包括比先前级别更详细的关键帧。 然后形成标题文件，其描述关键帧的有序集合，并且头文件与计算机可读存储器中的有序关键帧集合一起存储。 因此，用户可以根据来自相对较慢的存储器设备的视频图像的特定片段的偏好来识别和获得优化的检索。 该方法和装置同样适用于根据用户的喜好在一组多媒体文档中形成二进制大对象（“blob”）的索引顺序。