公开(公告)号：US07203380B2

公开(公告)日：2007-04-10

申请号：US09992617

申请日：2001-11-16

申请人： Patrick Chiu ,  Shingo Uchihashi ,  John S. Boreczky ,  Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn D. Wilcox

发明人： Patrick Chiu ,  Shingo Uchihashi ,  John S. Boreczky ,  Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn D. Wilcox

IPC分类号： G06K9/36

CPC分类号： H04N5/262

摘要： A method, system, and apparatus for easily creating a video collage from a video is provided. By segmenting the video into a set number of video segments and providing an interface for a user to select images which represent the video segments and insert the selected images into a video collage template, a video collage may be easily created in a short amount of time. The system is designed to assign values to the video inserted in a video collage and compact the video based on these values thereby creating a small file which may be easily stored or transmitted.

摘要翻译： 提供了一种用于从视频轻松创建视频拼贴的方法，系统和装置。 通过将视频分割成一定数量的视频片段并提供用于用户选择表示视频段的图像并将所选择的图像插入到视频拼贴模板中的界面，可以在短时间内容易地创建视频拼贴 。 该系统旨在为插入到视频拼贴中的视频分配值，并基于这些值压缩视频，从而创建可容易地存储或传输的小文件。

公开(公告)号：US06807361B1

公开(公告)日：2004-10-19

申请号：US09618533

申请日：2000-07-18

申请人： Andreas Girgensohn ,  John J. Doherty ,  Lynn D. Wilcox ,  John S. Boreczky ,  Patrick Chiu ,  Jonathan T. Foote

发明人： Andreas Girgensohn ,  John J. Doherty ,  Lynn D. Wilcox ,  John S. Boreczky ,  Patrick Chiu ,  Jonathan T. Foote

IPC分类号： H04N593

CPC分类号： G11B27/34 ,  G11B27/034 ,  G11B27/28

摘要： The present invention analyzes recorded video from a video camera to identify camera and object motion in the recorded video. Keyframes representative of clips of the recorded video are displayed on a user interface that allows a user to manipulate an order of the keyframes. Editing rules are then applied to the keyframes to intelligently splice together portions of the representative clips into a final output video.

摘要翻译： 本发明分析来自摄像机的记录视频，以识别记录的视频中的相机和物体运动。 表示记录视频的剪辑的关键帧显示在允许用户操纵关键帧顺序的用户界面上。 然后将编辑规则应用于关键帧，以将代表性剪辑的部分智能地拼接成最终输出视频。

公开(公告)号：US06839067B2

公开(公告)日：2005-01-04

申请号：US10205739

申请日：2002-07-26

申请人： Qiong Liu ,  Don Kimber ,  Lynn D. Wilcox ,  Jonathan T. Foote ,  John Boreczky ,  Patrick Chiu ,  Shingo Uchihashi ,  Hai Tao

发明人： Qiong Liu ,  Don Kimber ,  Lynn D. Wilcox ,  Jonathan T. Foote ,  John Boreczky ,  Patrick Chiu ,  Shingo Uchihashi ,  Hai Tao

IPC分类号： H04N7/14 ,  H04N7/15 ,  H04N7/173 ,  G09G5/00

CPC分类号： G08B13/19643 ,  H04N7/142 ,  H04N7/147 ,  H04N7/15

摘要： A method and apparatus for providing multi-resolution video to multiple users under hybrid human and automatic control. Initial environment and close-up images are captured using a first camera and a PTZ camera. The initial images are then stored in memory. Current environment and close-up images are captured and the an estimated difference between the initial and current images and the true image is determined. The estimated differences are weighted and compared and the stored images are updated. A close-up image is then provided to each user of the system. The close-up camera is then directed to a portion of the environment image having high distortion, and current environment and close-up images are captured again.

摘要翻译： 一种用于在混合人力和自动控制下向多个用户提供多分辨率视频的方法和装置。 使用第一台摄像机和一台PTZ摄像机拍摄初始环境和特写图像。 然后将初始图像存储在存储器中。 捕获当前环境和特写图像，并确定初始图像和当前图像与真实图像之间的估计差异。 估计的差异被加权和比较，并且存储的图像被更新。 然后将特写图像提供给系统的每个用户。 特写相机然后被引导到具有高失真的环境图像的一部分，并且再次捕获当前环境和特写图像。

公开(公告)号：US07712017B2

公开(公告)日：2010-05-04

申请号：US11361391

申请日：2006-02-24

申请人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

发明人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

IPC分类号： G06F17/00

CPC分类号： H04N1/32101 ,  H04N1/00318 ,  H04N1/0044 ,  H04N2201/3226 ,  H04N2201/3267 ,  H04N2201/3274

摘要： Video recordings of meetings and scanned paper documents are natural digital documents that come out of a meeting. These can be placed on the Internet for easy access, with links generated between them by matching scanned documents to a segment of the video referencing the scanned document. Furthermore, annotations made on the paper documents during the meeting can be extracted and used as indexes to the video. An orthonormal transform, such as a Digital Cosine Transform (DCT) is used to compare scanned documents to video frames.

摘要翻译： 会议和扫描纸文件的视频录制是出席会议的自然数字文件。 这些可以放在互联网上以便于访问，通过将扫描的文档与参考扫描文档的视频片段相匹配，在它们之间生成链接。 此外，可以提取会议期间在纸质文件上的注释，并将其用作视频的索引。 使用诸如数字余弦变换（DCT）的正交变换将扫描的文档与视频帧进行比较。

公开(公告)号：US07051271B1

公开(公告)日：2006-05-23

申请号：US09584205

申请日：2000-05-31

申请人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

发明人： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

IPC分类号： G06F17/00

CPC分类号： H04N1/32101 ,  H04N1/00318 ,  H04N1/0044 ,  H04N2201/3226 ,  H04N2201/3267 ,  H04N2201/3274

摘要： Video recordings of meetings and scanned paper documents are natural digital documents that come out of a meeting. These can be placed on the Internet for easy access, with links generated between them by matching scanned documents to a segment of the video referencing the scanned document. Furthermore, annotations made on the paper documents during the meeting can be extracted and used as indexes to the video. An orthonormal transform, such as a Digital Cosine Transform (DCT) is used to compare scanned documents to video frames.

摘要翻译： 会议和扫描纸文件的视频录制是出席会议的自然数字文件。 这些可以放在互联网上以便于访问，通过将扫描的文档与参考扫描文档的视频片段相匹配，在它们之间生成链接。 此外，可以提取会议期间在纸质文件上的注释，并将其用作视频的索引。 使用诸如数字余弦变换（DCT）的正交变换将扫描的文档与视频帧进行比较。

公开(公告)号：US06366296B1

公开(公告)日：2002-04-02

申请号：US09151285

申请日：1998-09-11

申请人： John S. Boreczky ,  Andreas Girgensohn ,  Jonathan T. Foote

发明人： John S. Boreczky ,  Andreas Girgensohn ,  Jonathan T. Foote

IPC分类号： G06F300

CPC分类号： G06F17/30852 ,  G06F3/0481 ,  G06F17/30787 ,  G06F17/30799 ,  H04N21/854 ,  Y10S707/99945 ,  Y10S707/99948 ,  Y10S715/968

摘要： A media browser, graphical user interface and method for browsing a media file wherein a user selects at least one feature in a media file and is provided with information regarding the existence of the selected feature in the media file. Based on the information, the user can identify and playback portions of interest in a media file. Features in a media file, such as a speaker's identity, applause, silence, motion, or video cuts, are preferably automatically time-wise evaluated in the media file using known methods. Metadata generated based on the time-wise feature evaluation are preferably mapped to confidence score values that represent a probability of a corresponding feature's existence in the media file. Confidence score information is preferably presented graphically to a user as part of a graphical user interface, and is used to interactively browse the media file.

摘要翻译： 用于浏览媒体文件的媒体浏览器，图形用户界面和方法，其中用户选择媒体文件中的至少一个特征并且提供有关媒体文件中所选特征的存在的信息。 基于该信息，用户可以识别并播放媒体文件中感兴趣的部分。 媒体文件中的特征，例如扬声器的身份，掌声，静音，运动或视频剪辑，优选地使用已知方法在媒体文件中自动地逐时评估。 基于时间特征评估生成的元数据优选地映射到表示媒体文件中对应特征存在概率的置信度分数值。 信心分数信息优选地以图形方式呈现给用户，作为图形用户界面的一部分，并且用于交互地浏览媒体文件。

公开(公告)号：US07260439B2

公开(公告)日：2007-08-21

申请号：US09985073

申请日：2001-11-01

申请人： Jonathan T. Foote ,  Matthew L. Cooper ,  Lynn D. Wilcox

发明人： Jonathan T. Foote ,  Matthew L. Cooper ,  Lynn D. Wilcox

IPC分类号： G06F17/00

CPC分类号： G11B27/28

摘要： A method of extracting audio excerpts comprises: segmenting audio data into a plurality of audio data segments; setting a fitness criteria for the plurality of audio data segments; analyzing the plurality of audio data segments based on the fitness criteria; and selecting one of the plurality of audio data segments that satisfies the fitness criteria. In various exemplary embodiments, the method of extracting audio excerpts further comprises associating the selected one of the plurality of audio data segments with video data. In such embodiments, associating the selected one of the plurality of audio data segments with video data may comprise associating the selected one of the plurality of audio data segments with a keyframe.

摘要翻译： 提取音频摘录的方法包括：将音频数据分割成多个音频数据段; 为所述多个音频数据段设置适合性标准; 基于适合性标准分析多个音频数据段; 以及选择满足适合度标准的多个音频数据段中的一个。 在各种示例性实施例中，提取音频摘录的方法还包括将所述多个音频数据段中的所选择的一个与视频数据相关联。 在这样的实施例中，将多个音频数据段中的所选择的一个与视频数据相关联可以包括将多个音频数据段中的所选择的一个与关键帧相关联。

公开(公告)号：US07246314B2

公开(公告)日：2007-07-17

申请号：US10859832

申请日：2004-06-03

申请人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

发明人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

IPC分类号： G06F15/00 ,  G06F14/00

CPC分类号： G06K9/00758 ,  G06F17/30814 ,  G06F17/30825 ,  G06F17/3084

摘要： Methods for interactive selecting video queries consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.

摘要翻译： 公开了用于交互式选择由用于视频相似性搜索的视频的训练图像组成的视频查询和用于显示相似性搜索的结果的方法。 用户选择视频中的时间间隔作为用于训练图像类统计模型的训练图像的查询定义。 时间间隔可以短到一帧，或者由不相交的片段或镜头组成。 从训练图像变换中提取的特征向量，计算由训练图像定义的图像类别的统计模型。 对于视频中的每个帧，从帧的变换中提取特征向量，并且使用特征向量和图像类统计模型来计算相似度度量。 相似性度量是从产生帧的高斯模型的可能性得出的。 然后以图形方式呈现相似性，这允许视频的时间结构可视化和浏览。 也可以为其他视频文件快速计算相似度，从而实现基于内容的检索。 介绍了具有交互式相似度测量功能的内容感知视频浏览器。 用于选择训练段的方法涉及通过表示视频持续时间的时间条来进行鼠标点击和拖动操作; 相似度结果在时间栏中显示为阴影。 另一种方法是选择视频的周期帧作为训练段的端点。

公开(公告)号：US06774917B1

公开(公告)日：2004-08-10

申请号：US09266558

申请日：1999-03-11

申请人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

发明人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

IPC分类号： G06F1500

CPC分类号： G06K9/00758 ,  G06F17/30814 ,  G06F17/30825 ,  G06F17/3084

摘要： Method for interactive selecting video consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.

摘要翻译： 公开了一种用于交互式选择视频组合的视频相似性搜索的视频的训练图像和用于显示相似性搜索的结果的方法。 用户选择视频中的时间间隔作为用于训练图像类统计模型的训练图像的查询定义。 时间间隔可以短到一帧，或者由不相交的片段或镜头组成。 从训练图像变换中提取的特征向量，计算由训练图像定义的图像类别的统计模型。 对于视频中的每个帧，从帧的变换中提取特征向量，并且使用特征向量和图像类统计模型来计算相似度度量。 相似性度量是从产生帧的高斯模型的可能性得出的。 然后以图形方式呈现相似性，这允许视频的时间结构可视化和浏览。 也可以为其他视频文件快速计算相似度，从而实现基于内容的检索。 介绍了具有交互式相似度测量功能的内容感知视频浏览器。 用于选择训练段的方法涉及通过表示视频持续时间的时间条来进行鼠标点击和拖动操作; 相似度结果在时间栏中显示为阴影。 另一种方法是选择视频的周期帧作为训练段的端点。

公开(公告)号：US06751354B2

公开(公告)日：2004-06-15

申请号：US09266637

申请日：1999-03-11

申请人： Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

发明人： Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

IPC分类号： G06K962

CPC分类号： G06K9/00758 ,  G06K9/6277 ,  G06K9/6297

摘要： Techniques for classifying video frames using statistical models of transform coefficients are disclosed. After optionally being decimated in time and space, image frames are transformed using a discrete cosine transform or Hadamard transform. The methods disclosed model image composition and operate on grayscale images. The resulting transform matrices are reduced using truncation, principal component analysis, or linear discriminant analysis to produce feature vectors. Feature vectors of training images for image classes are used to compute image class statistical models. Once image class statistical models are derived, individual frames are classified by the maximum likelihood resulting from the image class statistical models. Thus, the probabilities that a feature vector derived from a frame would be produced from each of the image class statistical models are computed. The frame is classified into the image class corresponding to the image class statistical model which produced the highest probability for the feature vector derived from the frame. Optionally, frame sequence information is taken into account by applying a hidden Markov model to represent image class transitions from the previous frame to the current frame. After computing all class probabilities for all frames in the video or sequence of frames using the image class statistical models and the image class transition probabilities, the final class is selected as having the maximum likelihood. Previous frames are selected in reverse order based upon their likelihood given determined current states.

摘要翻译： 公开了使用变换系数的统计模型对视频帧进行分类的技术。 在可选地在时间和空间中抽取后，使用离散余弦变换或Hadamard变换来转换图像帧。 该方法公开了模型图像组合，并对灰度图像进行操作。 所得到的变换矩阵使用截断，主成分分析或线性判别分析来减少以产生特征向量。 用于图像类的训练图像的特征向量用于计算图像类的统计模型。 一旦导出了图像类统计模型，则通过图像类统计模型产生的最大似然分类各个帧。 因此，计算从每个图像类统计模型产生从帧导出的特征向量的概率。 该帧被分类为对应于从帧产生的特征向量产生最高概率的图像类统计模型的图像类别。 可选地，通过应用隐马尔科夫模型来表示从先前帧到当前帧的图像类转换来考虑帧序列信息。 在使用图像类统计模型和图像类转换概率计算帧的视频或序列中的所有帧的所有类概率之后，选择最终类具有最大似然。 根据给定确定的当前状态的可能性，以相反的顺序选择先前的帧。