公开(公告)号：US06751354B2

公开(公告)日：2004-06-15

申请号：US09266637

申请日：1999-03-11

Applicant: Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

Inventor： Jonathan T. Foote ,  Lynn Wilcox ,  Andreas Girgensohn

IPC: G06K962

CPC classification number: G06K9/00758 ,  G06K9/6277 ,  G06K9/6297

Abstract: Techniques for classifying video frames using statistical models of transform coefficients are disclosed. After optionally being decimated in time and space, image frames are transformed using a discrete cosine transform or Hadamard transform. The methods disclosed model image composition and operate on grayscale images. The resulting transform matrices are reduced using truncation, principal component analysis, or linear discriminant analysis to produce feature vectors. Feature vectors of training images for image classes are used to compute image class statistical models. Once image class statistical models are derived, individual frames are classified by the maximum likelihood resulting from the image class statistical models. Thus, the probabilities that a feature vector derived from a frame would be produced from each of the image class statistical models are computed. The frame is classified into the image class corresponding to the image class statistical model which produced the highest probability for the feature vector derived from the frame. Optionally, frame sequence information is taken into account by applying a hidden Markov model to represent image class transitions from the previous frame to the current frame. After computing all class probabilities for all frames in the video or sequence of frames using the image class statistical models and the image class transition probabilities, the final class is selected as having the maximum likelihood. Previous frames are selected in reverse order based upon their likelihood given determined current states.

Abstract translation: 公开了使用变换系数的统计模型对视频帧进行分类的技术。 在可选地在时间和空间中抽取后，使用离散余弦变换或Hadamard变换来转换图像帧。 该方法公开了模型图像组合，并对灰度图像进行操作。 所得到的变换矩阵使用截断，主成分分析或线性判别分析来减少以产生特征向量。 用于图像类的训练图像的特征向量用于计算图像类的统计模型。 一旦导出了图像类统计模型，则通过图像类统计模型产生的最大似然分类各个帧。 因此，计算从每个图像类统计模型产生从帧导出的特征向量的概率。 该帧被分类为对应于从帧产生的特征向量产生最高概率的图像类统计模型的图像类别。 可选地，通过应用隐马尔科夫模型来表示从先前帧到当前帧的图像类转换来考虑帧序列信息。 在使用图像类统计模型和图像类转换概率计算帧的视频或序列中的所有帧的所有类概率之后，选择最终类具有最大似然。 根据给定确定的当前状态的可能性，以相反的顺序选择先前的帧。

公开(公告)号：US06535639B1

公开(公告)日：2003-03-18

申请号：US09267529

申请日：1999-03-12

Applicant: Shingo Uchihachi ,  Jonathan T. Foote ,  Lynn Wilcox

Inventor： Shingo Uchihachi ,  Jonathan T. Foote ,  Lynn Wilcox

IPC: G06K936

CPC classification number: G06K9/00711 ,  G06F17/30799 ,  G06F17/30843 ,  G06K9/00751 ,  G11B27/031 ,  G11B27/28 ,  Y10S707/99933 ,  Y10S707/99945

Abstract: A measure of importance is calculated for segmented parts of a video. The segmented parts are determined by segmenting the video into component shots and then merging by iteration the component shots based on similarity or other factors. Segmentation may also be determined by clustering frames of the video, and creating segments from the same cluster ID. The measure of importance is calculated based on a normalized weight of each segment and on length and rarity of each shot/segmented part. The importance measure may be utilized to generate a video summary by selecting the most important segments and generating representative frames for the selected segments. A thresholding process is applied to the importance score to provide a predetermined number or an appropriate number generated on the fly of shots or segments to be represented by frames. The representative frames are then packed into the video summary. The sizes of the frames to be packed are predetermined by their importance measure and adjusted according to space availability. Packing based on a grid and an exhaustive search of frame combinations to fill each row in the grid. A cost algorithm and a space-filling rule are utilized to determine the best fit of frames. The video summary may be presented on either a paper interface referencing or a web page linking the frames of the summary to points of the video.

Abstract translation: 为视频的分段部分计算重要度量度。 分割部分通过将视频分割成分量拍摄确定，然后通过迭代合并基于相似性或其他因素的分量拍摄。 分割也可以通过对视频的帧进行聚类来确定，并且从相同的集群ID创建分段。 重要性的度量是基于每个段的归一化权重和每个镜头/分割部分的长度和稀有度来计算的。 可以通过选择最重要的段并为所选择的段产生代表性帧来利用重要度量来产生视频摘要。 将阈值处理应用于重要性得分以提供在由帧表示的拍摄或片段的飞行中产生的预定数量或适当数量。 然后将代表性的框架包装在视频摘要中。 要包装的帧的大小由其重要性度量预先确定，并根据空间可用性进行调整。 基于网格的包装和对帧组合的详尽搜索来填充网格中的每一行。 使用成本算法和空格填充规则来确定帧的最佳拟合。 视频摘要可以呈现在纸张界面引用或将摘要的帧链接到视频的点的网页上。

公开(公告)号：US06219034B1

公开(公告)日：2001-04-17

申请号：US09028069

申请日：1998-02-23

Applicant: Kristofer E. Elbing ,  Jonathan T. Foote

Inventor： Kristofer E. Elbing ,  Jonathan T. Foote

IPC: G09G500

CPC classification number: G06F3/016 ,  G06F3/03543 ,  G06F3/0488 ,  G06F2203/014

Abstract: A computer system includes a computer processor, an operating system operative in connection with the computer processor, and a display responsive to the operating system. The system also has a pointing device that includes a position sensor and a tactile actuator. A pointing device driver is responsive to the position sensor, and the tactile actuator is responsive to the pointing device driver. A general-purpose application is responsive to the pointing device driver and to the operating system and in communication with the display, and the pointing device driver is also responsive to the general purpose application. The system further includes a profile that maps region changes associated with material displayed on the screen to tactile signals to be sent to the tactile actuator.

Abstract translation: 计算机系统包括计算机处理器，与计算机处理器连接操作的操作系统以及响应于操作系统的显示器。 该系统还具有包括位置传感器和触觉致动器的指示装置。 指示设备驱动器响应于位置传感器，并且触觉致动器响应于指示设备驱动器。 通用应用程序响应于指示设备驱动程序和操作系统并与显示器通信，并且指点设备驱动程序也响应于通用应用程序。 该系统还包括将与屏幕上显示的材料相关联的区域变化映射到要发送到触觉致动器的触觉信号的简档。

公开(公告)号：US09152230B1

公开(公告)日：2015-10-06

申请号：US12072502

申请日：2008-02-26

Applicant: Kristofer E. Elbing ,  Jonathan T. Foote

Inventor： Kristofer E. Elbing ,  Jonathan T. Foote

IPC: G06F3/0488 ,  G06F3/01

CPC classification number: G06F3/016 ,  G06F3/03543 ,  G06F3/0488 ,  G06F2203/014

Abstract: A computer system includes a computer processor, an operating system operative in connection with the computer processor, and a display responsive to the operating system. The system also has a pointing device that includes a position sensor and a tactile actuator. A pointing device driver is responsive to the position sensor, and the tactile actuator is responsive to the pointing device driver. A general-purpose application is responsive to the pointing device driver and to the operating system and in communication with the display, and the pointing device driver is also responsive to the general purpose application. The system further includes a profile that maps region changes associated with material displayed on the screen to tactile signals to be sent to the tactile actuator.

Abstract translation: 计算机系统包括计算机处理器，与计算机处理器连接操作的操作系统以及响应于操作系统的显示器。 该系统还具有包括位置传感器和触觉致动器的指示装置。 指示设备驱动器响应于位置传感器，并且触觉致动器响应于指示设备驱动器。 通用应用程序响应于指示设备驱动程序和操作系统并与显示器通信，并且指点设备驱动程序也响应于通用应用程序。 该系统还包括将与屏幕上显示的材料相关联的区域变化映射到要发送到触觉致动器的触觉信号的简档。

公开(公告)号：US08446456B2

公开(公告)日：2013-05-21

申请号：US11852224

申请日：2007-09-07

Applicant: Jonathan T. Foote ,  Donald G. Kimber ,  Xinding Sun ,  John E. Adcock

Inventor： Jonathan T. Foote ,  Donald G. Kimber ,  Xinding Sun ,  John E. Adcock

IPC: H04N7/00

CPC classification number: H04N5/268 ,  G06T7/97 ,  H04N5/262 ,  H04N5/2622

Abstract: Provides a system for detecting an intersection between more than one panoramic video sequence and detecting the orientation of the sequences forming the intersection. Video images and corresponding location data are received. If required, the images and location data is processed to ensure the images contain location data. An intersection between two paths is then derived from the video images by deriving a rough intersection between two images, determining a neighborhood for the two images, and dividing each image in the neighborhood into strips. An identifying value is derived from each strip to create a row of strip values which are then converted to the frequency domain. A distance measure is taken between strips in the frequency domain, and the intersection is determined from the images having the smallest distance measure between them. The orientation between the two paths may also be determined in the frequency domain by using the phases of signals representing the images in the Fourier domain or performing a circular cross correlation of two vectors representing the images.

Abstract translation: 提供一种用于检测多个全景视频序列之间的交点并检测形成交集的序列的取向的系统。 接收视频图像和对应的位置数据。 如果需要，处理图像和位置数据，以确保图像包含位置数据。 然后通过导出两个图像之间的粗略交点，确定两个图像的邻域，并将邻域中的每个图像分割成条，从而从视频图像导出两条路径之间的交点。 从每个条带导出识别值，以创建条带值行，然后将其转换为频域。 在频域中的带之间采取距离测量，并且从它们之间具有最小距离测量的图像确定交点。 也可以通过使用表示傅立叶域中的图像的信号的相位或者执行表示图像的两个向量的循环互相关来在频域中确定两个路径之间的取向。

公开(公告)号：US08126155B2

公开(公告)日：2012-02-28

申请号：US10612429

申请日：2003-07-02

Applicant: Qiong Liu ,  Donald G. Kimber ,  Jonathan T. Foote ,  Chunyuan Liao ,  John E. Adcock

Inventor： Qiong Liu ,  Donald G. Kimber ,  Jonathan T. Foote ,  Chunyuan Liao ,  John E. Adcock

IPC: H04R29/00 ,  H04N7/18 ,  H04N5/232

CPC classification number: H04H60/04

Abstract: An audio device management system (ADMS) manages remote audio devices via user selections in video links. The system enhances audio acquisition quality by receiving and processing human suggestions, forming customized two-way audio links according to user requests, and learning audio pickup strategies and camera management strategies from user operations. The ADMS control interface for a remote user provides a multi-window GUI that provides an overview window and selection display window. The ADMS provides users with more flexibility to enhance audio signals according to their needs and makes it more convenient to form customized two-way audio links without requiring users to remember a list of phone numbers. The ADMS also automatically manages available microphones for audio pickup based on microphone sound quality and the system's past experience when users monitor a structured audio environment without explicitly expressing their attentions in the video window.

Abstract translation: 音频设备管理系统（ADMS）通过视频链路中的用户选择来管理远程音频设备。 该系统通过接收和处理人的建议，根据用户要求形成定制的双向音频链接，以及从用户操作学习音频拾取策略和相机管理策略来提高音频采集质量。 远程用户的ADMS控制界面提供了一个提供概览窗口和选择显示窗口的多窗口GUI。 ADMS为用户提供了更多的灵活性，可根据需要增强音频信号，并使其更方便地形成定制的双向音频链接，而不需要用户记住电话号码列表。 ADMS还可以自动管理可用的麦克风，用于基于麦克风音质的音频拾音，以及当用户监视结构化音频环境而不明确表达其在视频窗口中的注意时，系统的过去体验。

公开(公告)号：US07712017B2

公开(公告)日：2010-05-04

申请号：US11361391

申请日：2006-02-24

Applicant: Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

Inventor： Patrick Chiu ,  Jonathan T. Foote ,  Andreas Girgensohn ,  John S. Boreczky

IPC: G06F17/00

CPC classification number: H04N1/32101 ,  H04N1/00318 ,  H04N1/0044 ,  H04N2201/3226 ,  H04N2201/3267 ,  H04N2201/3274

Abstract: Video recordings of meetings and scanned paper documents are natural digital documents that come out of a meeting. These can be placed on the Internet for easy access, with links generated between them by matching scanned documents to a segment of the video referencing the scanned document. Furthermore, annotations made on the paper documents during the meeting can be extracted and used as indexes to the video. An orthonormal transform, such as a Digital Cosine Transform (DCT) is used to compare scanned documents to video frames.

Abstract translation: 会议和扫描纸文件的视频录制是出席会议的自然数字文件。 这些可以放在互联网上以便于访问，通过将扫描的文档与参考扫描文档的视频片段相匹配，在它们之间生成链接。 此外，可以提取会议期间在纸质文件上的注释，并将其用作视频的索引。 使用诸如数字余弦变换（DCT）的正交变换将扫描的文档与视频帧进行比较。

公开(公告)号：US07710463B2

公开(公告)日：2010-05-04

申请号：US11345708

申请日：2006-02-02

Applicant: Jonathan T. Foote

Inventor： Jonathan T. Foote

IPC: H04N5/225 ,  H04N7/00 ,  H04N13/02

CPC classification number: G03B37/04 ,  G06T3/4038 ,  G06T5/006 ,  G06T7/33 ,  G06T7/97 ,  G06T2207/10012 ,  H04N5/23238 ,  H04N5/247 ,  H04N5/2628 ,  H04N13/128 ,  H04N13/239 ,  H04N13/243 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: A camera array captures plural component images which are combined into a single scene. In one embodiment, each camera of the array is a fixed digital camera. The images from each camera are warped to a common coordinate system and the disparity between overlapping images is reduced using disparity estimation techniques.

Abstract translation: 相机阵列捕获组合成单个场景的多个分量图像。 在一个实施例中，阵列的每个摄像机是固定数字照相机。 来自每个摄像机的图像变形到共同的坐标系，并且使用差异估计技术来减少重叠图像之间的差异。

公开(公告)号：US07337400B1

公开(公告)日：2008-02-26

申请号：US09835946

申请日：2001-04-16

Applicant: Kristofer E. Elbing ,  Jonathan T. Foote

Inventor： Kristofer E. Elbing ,  Jonathan T. Foote

IPC: G09G5/00 ,  G06K3/00

CPC classification number: G06F3/016 ,  G06F3/03543 ,  G06F3/0488 ,  G06F2203/014

Abstract: A computer system includes a computer processor, an operating system operative in connection with the computer processor, and a display responsive to the operating system. The system also has a pointing device that includes a position sensor and a tactile actuator. A pointing device driver is responsive to the position sensor, and the tactile actuator is responsive to the pointing device driver. A general-purpose application is responsive to the pointing device driver and to the operating system and in communication with the display, and the pointing device driver is also responsive to the general purpose application. The system further includes a profile that maps region changes associated with material displayed on the screen to tactile signals to be sent to the tactile actuator.

Abstract translation: 计算机系统包括计算机处理器，与计算机处理器连接操作的操作系统以及响应于操作系统的显示器。 该系统还具有包括位置传感器和触觉致动器的指示装置。 指示设备驱动器响应于位置传感器，并且触觉致动器响应于指示设备驱动器。 通用应用程序响应于指示设备驱动程序和操作系统并与显示器通信，并且指点设备驱动程序也响应于通用应用程序。 该系统还包括将与屏幕上显示的材料相关联的区域变化映射到要发送到触觉致动器的触觉信号的简档。

公开(公告)号：US07289138B2

公开(公告)日：2007-10-30

申请号：US10188667

申请日：2002-07-02

Applicant: Jonathan T. Foote ,  Donald Kimber ,  Xinding Sun ,  John Adcock

Inventor： Jonathan T. Foote ,  Donald Kimber ,  Xinding Sun ,  John Adcock

IPC: H04N7/00 ,  H04N7/18 ,  H04N9/47

CPC classification number: H04N5/268 ,  G06T7/97 ,  H04N5/262 ,  H04N5/2622

Abstract: Provides a system for detecting an intersection between more than one panoramic video sequence and detecting the orientation of the sequences forming the intersection. Video images and corresponding location data are received. If required, the images and location data is processed to ensure the images contain location data. An intersection between two paths is then derived from the video images by deriving a rough intersection between two images, determining a neighborhood for the two images, and dividing each image in the neighborhood into strips. An identifying value is derived from each strip to create a row of strip values which are then converted to the frequency domain. A distance measure is taken between strips in the frequency domain, and the intersection is determined from the images having the smallest distance measure between them. The orientation between the two paths may also be determined in the frequency domain by using the phases of signals representing the images in the Fourier domain or performing a circular cross correlation of two vectors representing the images.

Abstract translation: 提供一种用于检测多个全景视频序列之间的交点并检测形成交集的序列的取向的系统。 接收视频图像和对应的位置数据。 如果需要，处理图像和位置数据，以确保图像包含位置数据。 然后通过导出两个图像之间的粗略交点，确定两个图像的邻域，并将邻域中的每个图像分割成条，从而从视频图像导出两条路径之间的交点。 从每个条带导出识别值，以创建条带值行，然后将其转换为频域。 在频域中的带之间采取距离测量，并且从它们之间具有最小距离测量的图像确定交点。 也可以通过使用表示傅立叶域中的图像的信号的相位或者执行表示图像的两个向量的循环互相关来在频域中确定两个路径之间的取向。