公开(公告)号：US07246314B2

公开(公告)日：2007-07-17

申请号：US10859832

申请日：2004-06-03

申请人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

发明人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

IPC分类号： G06F15/00 ,  G06F14/00

CPC分类号： G06K9/00758 ,  G06F17/30814 ,  G06F17/30825 ,  G06F17/3084

摘要： Methods for interactive selecting video queries consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.

摘要翻译： 公开了用于交互式选择由用于视频相似性搜索的视频的训练图像组成的视频查询和用于显示相似性搜索的结果的方法。 用户选择视频中的时间间隔作为用于训练图像类统计模型的训练图像的查询定义。 时间间隔可以短到一帧，或者由不相交的片段或镜头组成。 从训练图像变换中提取的特征向量，计算由训练图像定义的图像类别的统计模型。 对于视频中的每个帧，从帧的变换中提取特征向量，并且使用特征向量和图像类统计模型来计算相似度度量。 相似性度量是从产生帧的高斯模型的可能性得出的。 然后以图形方式呈现相似性，这允许视频的时间结构可视化和浏览。 也可以为其他视频文件快速计算相似度，从而实现基于内容的检索。 介绍了具有交互式相似度测量功能的内容感知视频浏览器。 用于选择训练段的方法涉及通过表示视频持续时间的时间条来进行鼠标点击和拖动操作; 相似度结果在时间栏中显示为阴影。 另一种方法是选择视频的周期帧作为训练段的端点。

公开(公告)号：US06774917B1

公开(公告)日：2004-08-10

申请号：US09266558

申请日：1999-03-11

申请人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

发明人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

IPC分类号： G06F1500

CPC分类号： G06K9/00758 ,  G06F17/30814 ,  G06F17/30825 ,  G06F17/3084

摘要： Method for interactive selecting video consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.

摘要翻译： 公开了一种用于交互式选择视频组合的视频相似性搜索的视频的训练图像和用于显示相似性搜索的结果的方法。 用户选择视频中的时间间隔作为用于训练图像类统计模型的训练图像的查询定义。 时间间隔可以短到一帧，或者由不相交的片段或镜头组成。 从训练图像变换中提取的特征向量，计算由训练图像定义的图像类别的统计模型。 对于视频中的每个帧，从帧的变换中提取特征向量，并且使用特征向量和图像类统计模型来计算相似度度量。 相似性度量是从产生帧的高斯模型的可能性得出的。 然后以图形方式呈现相似性，这允许视频的时间结构可视化和浏览。 也可以为其他视频文件快速计算相似度，从而实现基于内容的检索。 介绍了具有交互式相似度测量功能的内容感知视频浏览器。 用于选择训练段的方法涉及通过表示视频持续时间的时间条来进行鼠标点击和拖动操作; 相似度结果在时间栏中显示为阴影。 另一种方法是选择视频的周期帧作为训练段的端点。

公开(公告)号：US06751354B2

公开(公告)日：2004-06-15

申请号：US09266637

申请日：1999-03-11

申请人： Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

发明人： Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

IPC分类号： G06K962

CPC分类号： G06K9/00758 ,  G06K9/6277 ,  G06K9/6297

摘要： Techniques for classifying video frames using statistical models of transform coefficients are disclosed. After optionally being decimated in time and space, image frames are transformed using a discrete cosine transform or Hadamard transform. The methods disclosed model image composition and operate on grayscale images. The resulting transform matrices are reduced using truncation, principal component analysis, or linear discriminant analysis to produce feature vectors. Feature vectors of training images for image classes are used to compute image class statistical models. Once image class statistical models are derived, individual frames are classified by the maximum likelihood resulting from the image class statistical models. Thus, the probabilities that a feature vector derived from a frame would be produced from each of the image class statistical models are computed. The frame is classified into the image class corresponding to the image class statistical model which produced the highest probability for the feature vector derived from the frame. Optionally, frame sequence information is taken into account by applying a hidden Markov model to represent image class transitions from the previous frame to the current frame. After computing all class probabilities for all frames in the video or sequence of frames using the image class statistical models and the image class transition probabilities, the final class is selected as having the maximum likelihood. Previous frames are selected in reverse order based upon their likelihood given determined current states.

摘要翻译： 公开了使用变换系数的统计模型对视频帧进行分类的技术。 在可选地在时间和空间中抽取后，使用离散余弦变换或Hadamard变换来转换图像帧。 该方法公开了模型图像组合，并对灰度图像进行操作。 所得到的变换矩阵使用截断，主成分分析或线性判别分析来减少以产生特征向量。 用于图像类的训练图像的特征向量用于计算图像类的统计模型。 一旦导出了图像类统计模型，则通过图像类统计模型产生的最大似然分类各个帧。 因此，计算从每个图像类统计模型产生从帧导出的特征向量的概率。 该帧被分类为对应于从帧产生的特征向量产生最高概率的图像类统计模型的图像类别。 可选地，通过应用隐马尔科夫模型来表示从先前帧到当前帧的图像类转换来考虑帧序列信息。 在使用图像类统计模型和图像类转换概率计算帧的视频或序列中的所有帧的所有类概率之后，选择最终类具有最大似然。 根据给定确定的当前状态的可能性，以相反的顺序选择先前的帧。

公开(公告)号：US07712017B2

公开(公告)日：2010-05-04

申请号：US11361391

申请日：2006-02-24

申请人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

发明人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

IPC分类号： G06F17/00

CPC分类号： H04N1/32101 ,  H04N1/00318 ,  H04N1/0044 ,  H04N2201/3226 ,  H04N2201/3267 ,  H04N2201/3274

摘要： Video recordings of meetings and scanned paper documents are natural digital documents that come out of a meeting. These can be placed on the Internet for easy access, with links generated between them by matching scanned documents to a segment of the video referencing the scanned document. Furthermore, annotations made on the paper documents during the meeting can be extracted and used as indexes to the video. An orthonormal transform, such as a Digital Cosine Transform (DCT) is used to compare scanned documents to video frames.

摘要翻译： 会议和扫描纸文件的视频录制是出席会议的自然数字文件。 这些可以放在互联网上以便于访问，通过将扫描的文档与参考扫描文档的视频片段相匹配，在它们之间生成链接。 此外，可以提取会议期间在纸质文件上的注释，并将其用作视频的索引。 使用诸如数字余弦变换（DCT）的正交变换将扫描的文档与视频帧进行比较。

公开(公告)号：US07051271B1

公开(公告)日：2006-05-23

申请号：US09584205

申请日：2000-05-31

申请人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

发明人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

IPC分类号： G06F17/00

CPC分类号： H04N1/32101 ,  H04N1/00318 ,  H04N1/0044 ,  H04N2201/3226 ,  H04N2201/3267 ,  H04N2201/3274

摘要： Video recordings of meetings and scanned paper documents are natural digital documents that come out of a meeting. These can be placed on the Internet for easy access, with links generated between them by matching scanned documents to a segment of the video referencing the scanned document. Furthermore, annotations made on the paper documents during the meeting can be extracted and used as indexes to the video. An orthonormal transform, such as a Digital Cosine Transform (DCT) is used to compare scanned documents to video frames.

摘要翻译： 会议和扫描纸文件的视频录制是出席会议的自然数字文件。 这些可以放在互联网上以便于访问，通过将扫描的文档与参考扫描文档的视频片段相匹配，在它们之间生成链接。 此外，可以提取会议期间在纸质文件上的注释，并将其用作视频的索引。 使用诸如数字余弦变换（DCT）的正交变换将扫描的文档与视频帧进行比较。

公开(公告)号：US06366296B1

公开(公告)日：2002-04-02

申请号：US09151285

申请日：1998-09-11

申请人： John S. Boreczky ,  Andreas Girgensohn ,  Jonathan T. Foote

发明人： John S. Boreczky ,  Andreas Girgensohn ,  Jonathan T. Foote

IPC分类号： G06F300

CPC分类号： G06F17/30852 ,  G06F3/0481 ,  G06F17/30787 ,  G06F17/30799 ,  H04N21/854 ,  Y10S707/99945 ,  Y10S707/99948 ,  Y10S715/968

摘要： A media browser, graphical user interface and method for browsing a media file wherein a user selects at least one feature in a media file and is provided with information regarding the existence of the selected feature in the media file. Based on the information, the user can identify and playback portions of interest in a media file. Features in a media file, such as a speaker's identity, applause, silence, motion, or video cuts, are preferably automatically time-wise evaluated in the media file using known methods. Metadata generated based on the time-wise feature evaluation are preferably mapped to confidence score values that represent a probability of a corresponding feature's existence in the media file. Confidence score information is preferably presented graphically to a user as part of a graphical user interface, and is used to interactively browse the media file.

摘要翻译： 用于浏览媒体文件的媒体浏览器，图形用户界面和方法，其中用户选择媒体文件中的至少一个特征并且提供有关媒体文件中所选特征的存在的信息。 基于该信息，用户可以识别并播放媒体文件中感兴趣的部分。 媒体文件中的特征，例如扬声器的身份，掌声，静音，运动或视频剪辑，优选地使用已知方法在媒体文件中自动地逐时评估。 基于时间特征评估生成的元数据优选地映射到表示媒体文件中对应特征存在概率的置信度分数值。 信心分数信息优选地以图形方式呈现给用户，作为图形用户界面的一部分，并且用于交互地浏览媒体文件。

公开(公告)号：US07203380B2

公开(公告)日：2007-04-10

申请号：US09992617

申请日：2001-11-16

申请人： Patrick Chiu ,  Shingo Uchihashi ,  John S. Boreczky ,  Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn D. Wilcox

发明人： Patrick Chiu ,  Shingo Uchihashi ,  John S. Boreczky ,  Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn D. Wilcox

IPC分类号： G06K9/36

CPC分类号： H04N5/262

摘要： A method, system, and apparatus for easily creating a video collage from a video is provided. By segmenting the video into a set number of video segments and providing an interface for a user to select images which represent the video segments and insert the selected images into a video collage template, a video collage may be easily created in a short amount of time. The system is designed to assign values to the video inserted in a video collage and compact the video based on these values thereby creating a small file which may be easily stored or transmitted.

摘要翻译： 提供了一种用于从视频轻松创建视频拼贴的方法，系统和装置。 通过将视频分割成一定数量的视频片段并提供用于用户选择表示视频段的图像并将所选择的图像插入到视频拼贴模板中的界面，可以在短时间内容易地创建视频拼贴 。 该系统旨在为插入到视频拼贴中的视频分配值，并基于这些值压缩视频，从而创建可容易地存储或传输的小文件。

公开(公告)号：US06807361B1

公开(公告)日：2004-10-19

申请号：US09618533

申请日：2000-07-18

申请人： Andreas Girgensohn ,  John J. Doherty ,  Lynn D. Wilcox ,  John S. Boreczky ,  Patrick Chiu ,  Jonathan T. Foote

发明人： Andreas Girgensohn ,  John J. Doherty ,  Lynn D. Wilcox ,  John S. Boreczky ,  Patrick Chiu ,  Jonathan T. Foote

IPC分类号： H04N593

CPC分类号： G11B27/34 ,  G11B27/034 ,  G11B27/28

摘要： The present invention analyzes recorded video from a video camera to identify camera and object motion in the recorded video. Keyframes representative of clips of the recorded video are displayed on a user interface that allows a user to manipulate an order of the keyframes. Editing rules are then applied to the keyframes to intelligently splice together portions of the representative clips into a final output video.

摘要翻译： 本发明分析来自摄像机的记录视频，以识别记录的视频中的相机和物体运动。 表示记录视频的剪辑的关键帧显示在允许用户操纵关键帧顺序的用户界面上。 然后将编辑规则应用于关键帧，以将代表性剪辑的部分智能地拼接成最终输出视频。

公开(公告)号：US08089563B2

公开(公告)日：2012-01-03

申请号：US11324557

申请日：2006-01-03

申请人： Andreas Girgensohn ,  Frank M. Shipman ,  Lynn Wilcox ,  Donald G. Kimber

发明人： Andreas Girgensohn ,  Frank M. Shipman ,  Lynn Wilcox ,  Donald G. Kimber

IPC分类号： H04N5/14 ,  H04N7/18 ,  G06K9/34

CPC分类号： G06F17/30811 ,  G06F17/30843 ,  G06F17/30852 ,  G06K9/00711 ,  H04N7/181

摘要： Techniques for generating a storyboard are disclosed. In one embodiment of the invention the storyboard is comprised of videos from one or more cameras based on the identification of activity in the video. Various embodiments of the invention include an assessment of the importance of the activity, the creation of a storyboard presentation based on importance and interaction techniques for seeing more details or alternate views of the video. In one embodiment, motion detection is used to determine activity in one or more synchronized video streams. Periods of activity are recognized and assigned importance assessments based on the activity, important locations in the video streams, and events from other sensors. In different embodiments, the interface consists of a storyboard and a map.

摘要翻译： 公开了生成故事板的技术。 在本发明的一个实施例中，故事板由来自一个或多个相机的视频组成，基于视频中活动的识别。 本发明的各种实施例包括评估活动的重要性，基于重要性的创建故事板呈现以及用于观看视频的更多细节或替代视图的交互技术。 在一个实施例中，运动检测用于确定一个或多个同步视频流中的活动。 根据活动，视频流中的重要位置以及来自其他传感器的事件，识别和分配活动的时间段并进行重要性评估。 在不同的实施例中，界面由故事板和地图组成。

公开(公告)号：US07996771B2

公开(公告)日：2011-08-09

申请号：US11324971

申请日：2006-01-03

申请人： Andreas Girgensohn ,  Frank M. Shipman ,  Lynn Wilcox

发明人： Andreas Girgensohn ,  Frank M. Shipman ,  Lynn Wilcox

IPC分类号： G06F3/00

CPC分类号： G06F17/3079 ,  G06F17/30802 ,  G06F17/30843 ,  G08B13/19682

摘要： Techniques for generating timelines and event logs from one or more fixed-position cameras based on the identification of activity in the video are presented. Various embodiments of the invention include an assessment of the importance of the activity, the creation of a timeline identifying events of interest, and interaction techniques for seeing more details of an event or alternate views of the video. In one embodiment, motion detection is used to determine activity in one or more synchronized video streams. In another embodiment, events are determined based on periods of activity and assigned importance assessments based on the activity, important locations in the video streams, and events from other sensors. In different embodiments, the interface consists of a timeline, event log, and map.

摘要翻译： 提出了基于视频中的活动识别从一个或多个固定位置摄像机生成时间线和事件日志的技术。 本发明的各种实施例包括评估活动的重要性，创建识别感兴趣事件的时间线以及用于查看视频的事件或替代视图的更多细节的交互技术。 在一个实施例中，运动检测用于确定一个或多个同步视频流中的活动。 在另一个实施例中，基于活动的周期和基于活动的重要性评估，视频流中的重要位置以及来自其他传感器的事件来确定事件。 在不同的实施例中，接口由时间线，事件日志和地图组成。