公开(公告)号：US07246314B2

公开(公告)日：2007-07-17

申请号：US10859832

申请日：2004-06-03

申请人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

发明人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

IPC分类号： G06F15/00 ,  G06F14/00

CPC分类号： G06K9/00758 ,  G06F17/30814 ,  G06F17/30825 ,  G06F17/3084

摘要： Methods for interactive selecting video queries consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.

摘要翻译： 公开了用于交互式选择由用于视频相似性搜索的视频的训练图像组成的视频查询和用于显示相似性搜索的结果的方法。 用户选择视频中的时间间隔作为用于训练图像类统计模型的训练图像的查询定义。 时间间隔可以短到一帧，或者由不相交的片段或镜头组成。 从训练图像变换中提取的特征向量，计算由训练图像定义的图像类别的统计模型。 对于视频中的每个帧，从帧的变换中提取特征向量，并且使用特征向量和图像类统计模型来计算相似度度量。 相似性度量是从产生帧的高斯模型的可能性得出的。 然后以图形方式呈现相似性，这允许视频的时间结构可视化和浏览。 也可以为其他视频文件快速计算相似度，从而实现基于内容的检索。 介绍了具有交互式相似度测量功能的内容感知视频浏览器。 用于选择训练段的方法涉及通过表示视频持续时间的时间条来进行鼠标点击和拖动操作; 相似度结果在时间栏中显示为阴影。 另一种方法是选择视频的周期帧作为训练段的端点。

公开(公告)号：US06774917B1

公开(公告)日：2004-08-10

申请号：US09266558

申请日：1999-03-11

申请人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

发明人： Jonathan T. Foote ,  Andreas Girgensohn ,  Lynn Wilcox

IPC分类号： G06F1500

CPC分类号： G06K9/00758 ,  G06F17/30814 ,  G06F17/30825 ,  G06F17/3084

摘要： Method for interactive selecting video consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.

摘要翻译： 公开了一种用于交互式选择视频组合的视频相似性搜索的视频的训练图像和用于显示相似性搜索的结果的方法。 用户选择视频中的时间间隔作为用于训练图像类统计模型的训练图像的查询定义。 时间间隔可以短到一帧，或者由不相交的片段或镜头组成。 从训练图像变换中提取的特征向量，计算由训练图像定义的图像类别的统计模型。 对于视频中的每个帧，从帧的变换中提取特征向量，并且使用特征向量和图像类统计模型来计算相似度度量。 相似性度量是从产生帧的高斯模型的可能性得出的。 然后以图形方式呈现相似性，这允许视频的时间结构可视化和浏览。 也可以为其他视频文件快速计算相似度，从而实现基于内容的检索。 介绍了具有交互式相似度测量功能的内容感知视频浏览器。 用于选择训练段的方法涉及通过表示视频持续时间的时间条来进行鼠标点击和拖动操作; 相似度结果在时间栏中显示为阴影。 另一种方法是选择视频的周期帧作为训练段的端点。

公开(公告)号：US07401351B2

公开(公告)日：2008-07-15

申请号：US09738905

申请日：2000-12-14

申请人： John Boreczky ,  Andreas Girgensohn ,  Lynn Wilcox ,  William N. Schilit

发明人： John Boreczky ,  Andreas Girgensohn ,  Lynn Wilcox ,  William N. Schilit

IPC分类号： H04N7/173

CPC分类号： H04N21/431 ,  H04N7/17336 ,  H04N21/234363 ,  H04N21/234381 ,  H04N21/6587 ,  H04N21/8455

摘要： A system is provided where a number of representative video snapshots are displayed on a timeline indicating a position to jump to in a video using control buttons, such as fast forward, reverse, or a timeline scroll bar. The video snapshots are obtained by opening a low resolution connection to a video server to receive forward looking video segments. The forward looking video segments are summarized and the representative video snapshots are selected and displayed to provide a reference frame for the user to select a timeline position, or snapshots directly linked to positions in the higher resolution video. Backward looking video segments may be similarly summarized from the low resolution connection, or alternatively from video previously played. As the video progresses, new forward looking video segments are received and new representative video snapshots are summarized and displayed in close proximity with the video being played.

摘要翻译： 提供了一种系统，其中使用诸如快进，反向或时间轴滚动条的控制按钮在时间线上显示许多代表性的视频快照，其指示在视频中跳转的位置。 视频快照是通过打开与视频服务器的低分辨率连接来接收前视视频片段而获得的。 总结前视视频片段并且选择和显示代表性视频快照以提供用于用户选择时间线位置的参考帧或者直接链接到较高分辨率视频中的位置的快照。 可以从低分辨率连接或者替代地从先前播放的视频来类似地总结向后看的视频片段。 随着视频的进行，接收到新的前瞻性视频片段，并且新的代表性视频快照被汇总并显示在正在播放的视频附近。

公开(公告)号：US20060288288A1

公开(公告)日：2006-12-21

申请号：US11324971

申请日：2006-01-03

申请人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox

发明人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox

IPC分类号： G11B27/00

CPC分类号： G06F17/3079 ,  G06F17/30802 ,  G06F17/30843 ,  G08B13/19682

摘要： Techniques for generating timelines and event logs from one or more fixed-position cameras based on the identification of activity in the video are presented. Various embodiments of the invention include an assessment of the importance of the activity, the creation of a timeline identifying events of interest, and interaction techniques for seeing more details of an event or alternate views of the video. In one embodiment, motion detection is used to determine activity in one or more synchronized video streams. In another embodiment, events are determined based on periods of activity and assigned importance assessments based on the activity, important locations in the video streams, and events from other sensors. In different embodiments, the interface consists of a timeline, event log, and map.

摘要翻译： 提出了基于视频中的活动识别从一个或多个固定位置摄像机生成时间线和事件日志的技术。 本发明的各种实施例包括评估活动的重要性，创建识别感兴趣事件的时间线以及用于查看视频的事件或替代视图的更多细节的交互技术。 在一个实施例中，运动检测用于确定一个或多个同步视频流中的活动。 在另一个实施例中，基于活动的周期和基于活动的重要性评估，视频流中的重要位置以及来自其他传感器的事件来确定事件。 在不同的实施例中，接口由时间线，事件日志和地图组成。

公开(公告)号：US20060284976A1

公开(公告)日：2006-12-21

申请号：US11324355

申请日：2006-01-03

申请人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox ,  Donald Kimber

发明人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox ,  Donald Kimber

IPC分类号： H04N7/18 ,  G06K9/62 ,  H04N5/14

CPC分类号： G06F17/30811

摘要： Techniques for generating action keyframes for a fixed-position camera based on the identification of activity in the video, an assessment of the importance of the activity, object recognition in the video, and interaction techniques for seeing more details of the video are presented. In different embodiments of the invention, the importance of activity is determined based on the amount of activity, important locations in the video streams, detected features such as faces, and events from other sensors.

公开(公告)号：US20050002647A1

公开(公告)日：2005-01-06

申请号：US10612428

申请日：2003-07-02

申请人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox

发明人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox

IPC分类号： G06F17/30 ,  G11B27/034 ,  G11B27/10 ,  G11B27/34 ,  H04N5/783 ,  H04N5/92

CPC分类号： G11B27/105 ,  G11B27/034 ,  G11B27/34 ,  G11B2220/2562

摘要： A hypervideo summary comprised of multiple levels of related content and appropriate navigational links can be automatically generated from a media file such as a linear video. A number of algorithms and selection criteria can be used to modify how such a summary is generated. Viewers of an automatically-generated hypervideo summary can interactively select the amount of detail displayed for each portion of the summary. This selection can be done by following explicit navigational links, or by changing between media channels that are mapped to the various levels of related content. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.

摘要翻译： 可以从诸如线性视频的媒体文件自动地生成包括多级相关内容和适当导航链接的超视频摘要。 可以使用许多算法和选择标准来修改如何生成摘要。 自动生成的超视频摘要的查看者可以交互地选择为摘要的每个部分显示的详细信息量。 该选择可以通过以下显式导航链接，或者通过在映射到相关内容的各个级别的媒体频道之间进行改变来完成。 本说明书不是对本发明的完整描述或限制本发明的范围。 本发明的其它特征，方面和目的可以通过对说明书，附图和权利要求的评述来获得。

公开(公告)号：US20110197166A1

公开(公告)日：2011-08-11

申请号：US12701466

申请日：2010-02-05

申请人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox

发明人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox

IPC分类号： G06F17/30 ,  G06F3/048

CPC分类号： G06F16/90 ,  G06F16/93

摘要： Systems and methods recommend documents or directories in an enterprise context by analyzing use proximity in the organizational hierarchy to find similar users. Evidence may be used from different sources to gage the degree of interest a user may have in a document. Such pieces of evidence may include viewing page thumbnails, viewing the document online, or printing, saving, or bookmarking the document. To make managing these different sources of evidence from users related by different degrees tractable, a model may be created where evidence values decay by different amounts over time and are combined with different weights. Additionally, these systems and methods also can make use of the directory structure of the document space to recommend directories as well as individual documents.

摘要翻译： 系统和方法通过分析组织层次结构中的使用接近度来查找类似的用户来推荐企业上下文中的文档或目录。 可以从不同来源使用证据来衡量用户在文档中可能具有的兴趣程度。 这些证据可能包括查看页面缩略图，在线查看文档，或打印，保存或加书签。 为了使用不同程度的用户管理这些不同来源的证据，可以创建一个模型，其中证据值随着时间的推移不同的数量衰减并与不同的权重组合。 此外，这些系统和方法还可以利用文档空间的目录结构来推荐目录以及单个文档。

公开(公告)号：US20090313267A1

公开(公告)日：2009-12-17

申请号：US12137718

申请日：2008-06-12

申请人： Andreas Girgensohn ,  Frank Shipman, III ,  Lynn Wilcox

发明人： Andreas Girgensohn ,  Frank Shipman, III ,  Lynn Wilcox

IPC分类号： G06F17/30

CPC分类号： G06F17/30058

摘要： An adaptive, interactive visual workspace for viewing groups of files based on their relationships. Relationships of files are visualized using iterative refinement of categories through a direct-manipulation graph-based layout. The visual workspace starts with a fully connected graph linking thumbnail images of related files that is then partitioned into neighborhoods in response to a user creating file stacks corresponding to different categories. Normalized spring lengths improve the overall quality of the layout. Different modes for membership in neighborhoods avoid confusing motion of files and help a user to manually organize the workspace. Additionally, retrieved files can be added without having to significantly move the previous files. Different visualization techniques indicate which files are related to each other. An elliptical-shaped window displaying rings of elliptic thumbnail images presents a group of files in an easily visible and space-efficient way. Different zoom rates are used for file location, and surrogate sizes allow users to increase the separation between files while still increasing the surrogate sizes.

摘要翻译： 一个自适应的交互式视觉工作区，用于根据他们的关系来查看文件组。 文件的关系通过直接操作图形布局的类别迭代细化进行可视化。 可视化工作区以完全连接的图形开始，将图形相关联的文件链接起来，然后将其分成邻域，以响应用户创建对应于不同类别的文件堆栈。 标准化的弹簧长度提高了布局的整体质量。 用于居民区成员身份的不同模式可避免混淆文件运动，并帮助用户手动组织工作区。 此外，可以添加检索到的文件，而不必显着移动以前的文件。 不同的可视化技术指出哪些文件是相互关联的。 显示椭圆缩略图像环的椭圆形窗口以易于看见和节省空间的方式呈现一组文件。 文件位置使用不同的缩放比例，替代尺寸允许用户增加文件之间的间隔，同时仍然增加代理大小。

公开(公告)号：US20060284978A1

公开(公告)日：2006-12-21

申请号：US11324557

申请日：2006-01-03

申请人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox ,  Donald Kimber

发明人： Andreas Girgensohn ,  Frank Shipman ,  Lynn Wilcox ,  Donald Kimber

IPC分类号： H04N7/18 ,  H04N5/14 ,  H04N9/64 ,  H04N9/47

CPC分类号： G06F17/30811 ,  G06F17/30843 ,  G06F17/30852 ,  G06K9/00711 ,  H04N7/181

摘要： Techniques for generating a storyboard are disclosed. In one embodiment of the invention the storyboard is comprised of videos from one or more cameras based on the identification of activity in the video. Various embodiments of the invention include an assessment of the importance of the activity, the creation of a storyboard presentation based on importance and interaction techniques for seeing more details or alternate views of the video. In one embodiment, motion detection is used to determine activity in one or more synchronized video streams. Periods of activity are recognized and assigned importance assessments based on the activity, important locations in the video streams, and events from other sensors. In different embodiments, the interface consists of a storyboard and a map.

摘要翻译： 公开了生成故事板的技术。 在本发明的一个实施例中，故事板由来自一个或多个相机的视频组成，基于视频中活动的识别。 本发明的各种实施例包括评估活动的重要性，基于重要性的创建故事板呈现以及用于观看视频的更多细节或替代视图的交互技术。 在一个实施例中，运动检测用于确定一个或多个同步视频流中的活动。 根据活动，视频流中的重要位置以及来自其他传感器的事件，识别和分配活动的时间段并进行重要性评估。 在不同的实施例中，界面由故事板和地图组成。

公开(公告)号：US06751354B2

公开(公告)日：2004-06-15

申请号：US09266637

申请日：1999-03-11

申请人： Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

发明人： Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

IPC分类号： G06K962

CPC分类号： G06K9/00758 ,  G06K9/6277 ,  G06K9/6297

摘要： Techniques for classifying video frames using statistical models of transform coefficients are disclosed. After optionally being decimated in time and space, image frames are transformed using a discrete cosine transform or Hadamard transform. The methods disclosed model image composition and operate on grayscale images. The resulting transform matrices are reduced using truncation, principal component analysis, or linear discriminant analysis to produce feature vectors. Feature vectors of training images for image classes are used to compute image class statistical models. Once image class statistical models are derived, individual frames are classified by the maximum likelihood resulting from the image class statistical models. Thus, the probabilities that a feature vector derived from a frame would be produced from each of the image class statistical models are computed. The frame is classified into the image class corresponding to the image class statistical model which produced the highest probability for the feature vector derived from the frame. Optionally, frame sequence information is taken into account by applying a hidden Markov model to represent image class transitions from the previous frame to the current frame. After computing all class probabilities for all frames in the video or sequence of frames using the image class statistical models and the image class transition probabilities, the final class is selected as having the maximum likelihood. Previous frames are selected in reverse order based upon their likelihood given determined current states.

摘要翻译： 公开了使用变换系数的统计模型对视频帧进行分类的技术。 在可选地在时间和空间中抽取后，使用离散余弦变换或Hadamard变换来转换图像帧。 该方法公开了模型图像组合，并对灰度图像进行操作。 所得到的变换矩阵使用截断，主成分分析或线性判别分析来减少以产生特征向量。 用于图像类的训练图像的特征向量用于计算图像类的统计模型。 一旦导出了图像类统计模型，则通过图像类统计模型产生的最大似然分类各个帧。 因此，计算从每个图像类统计模型产生从帧导出的特征向量的概率。 该帧被分类为对应于从帧产生的特征向量产生最高概率的图像类统计模型的图像类别。 可选地，通过应用隐马尔科夫模型来表示从先前帧到当前帧的图像类转换来考虑帧序列信息。 在使用图像类统计模型和图像类转换概率计算帧的视频或序列中的所有帧的所有类概率之后，选择最终类具有最大似然。 根据给定确定的当前状态的可能性，以相反的顺序选择先前的帧。