公开(公告)号：US07433495B2

公开(公告)日：2008-10-07

申请号：US11042453

申请日：2005-01-25

申请人： Yong Rui ,  Yunqiang Chen

发明人： Yong Rui ,  Yunqiang Chen

IPC分类号： G06K9/00 ,  G06K9/38 ,  G06K9/62 ,  G06K9/66 ,  G06K9/74

CPC分类号： G06K9/00234 ,  G06T7/251 ,  G06T2207/10016 ,  G06T2207/30196 ,  G06T2207/30201

摘要： Automatic detection and tracking of multiple individuals includes receiving a frame of video and/or audio content and identifying a candidate area for a new face region in the frame. One or more hierarchical verification levels are used to verify whether a human face is in the candidate area, and an indication made that the candidate area includes a face if the one or more hierarchical verification levels verify that a human face is in the candidate area. A plurality of audio and/or video cues are used to track each verified face in the video content from frame to frame.

公开(公告)号：US07428000B2

公开(公告)日：2008-09-23

申请号：US10608313

申请日：2003-06-26

申请人： Ross Cutler ,  Yong Rui ,  Anoop Gupta

发明人： Ross Cutler ,  Yong Rui ,  Anoop Gupta

IPC分类号： H04N7/14

CPC分类号： H04N7/15 ,  H04N7/152 ,  H04N7/155

摘要： A system and method for teleconferencing and recording of meetings. The system uses a variety of capture devices (a novel 360° camera, a whiteboard camera, a presenter view camera, a remote view camera, and a microphone array) to provide a rich experience for people who want to participate in a meeting from a distance. The system is also combined with speaker clustering, spatial indexing, and time compression to provide a rich experience for people who miss a meeting and want to watch it afterward.

摘要翻译： 电话会议和会议记录的系统和方法。 该系统使用各种捕获设备（新颖的360度相机，白板摄像头，演示者相机，遥控摄像头和麦克风阵列），为希望参加会议的人们提供丰富的体验 距离。 该系统还结合扬声器群集，空间索引和时间压缩，为错过会议并希望观看的人们提供丰富的体验。

公开(公告)号：US07403894B2

公开(公告)日：2008-07-22

申请号：US11081118

申请日：2005-03-15

申请人： Yong Rui ,  Anoop Gupta ,  Alejandro Acero

发明人： Yong Rui ,  Anoop Gupta ,  Alejandro Acero

IPC分类号： G10L21/00

CPC分类号： G06F17/30787 ,  G06F17/30743 ,  G06F17/30749 ,  G06F17/30758 ,  G06F17/30843 ,  G06K9/00711

摘要： Audio/video programming content is made available to a receiver from a content provider, and meta data is made available to the receiver from a meta data provider. The meta data corresponds to the programming content, and identifies, for each of multiple portions of the programming content, an indicator of a likelihood that the portion is an exciting portion of the content. In one implementation, the meta data includes probabilities that segments of a baseball program are exciting, and is generated by analyzing the audio data of the baseball program for both excited speech and baseball hits. The meta data can then be used to generate a summary for the baseball program.

摘要翻译： 音频/视频节目内容从内容提供商可用于接收者，并且元数据从元数据提供者向接收者提供。 元数据对应于节目内容，并且针对节目内容的多个部分中的每一个识别该部分是内容的激动部分的可能性的指示符。 在一个实现中，元数据包括棒球节目的节段令人兴奋的概率，并且通过分析用于激发的语音和棒球命中的棒球节目的音频数据而产生。 然后可以使用元数据来生成棒球程序的摘要。

公开(公告)号：US20070186171A1

公开(公告)日：2007-08-09

申请号：US11351049

申请日：2006-02-09

申请人： Sasa Junuzovic ,  Prasun Dewan ,  Yong Rui

发明人： Sasa Junuzovic ,  Prasun Dewan ,  Yong Rui

IPC分类号： G06F3/00

CPC分类号： G06F3/0481 ,  G06Q10/10

摘要： Techniques are provided for indicating workspace awareness using one or more of a write shadow, a read shadow, and/or a shadowbar providing an indication of operations performed at associated locations by various users accessing a same document. A write shadow may be used to indicate a position in a document being modified by a user. A read shadow may be used to indicate a position being viewed by a user. A shadowbar may be used to indicate areas of overlap among users with a shading and coloring indicative of a degree of overlap.

摘要翻译： 提供了用于使用访问相同文档的各种用户的在相关联位置处执行的操作的指示的写影，读影和/或阴影栏中的一个或多个来指示工作空间感知的技术。 可以使用写入阴影来指示由用户修改的文档中的位置。 读影子可以用于指示用户正在观看的位置。 阴影栏可以用于指示具有指示重叠度的阴影和着色的用户之间的重叠区域。

公开(公告)号：US07254241B2

公开(公告)日：2007-08-07

申请号：US11190241

申请日：2005-07-26

申请人： Yong Rui ,  Dinei Florencio

发明人： Yong Rui ,  Dinei Florencio

IPC分类号： H04R3/00

CPC分类号： H04R3/005 ,  G10L21/0272 ,  G10L2021/02165

摘要： A system and process for finding the location of a sound source using direct approaches having weighting factors that mitigate the effect of both correlated and reverberation noise is presented. When more than two microphones are used, the traditional time-delay-of-arrival (TDOA) based sound source localization (SSL) approach involves two steps. The first step computes TDOA for each microphone pair, and the second step combines these estimates. This two-step process discards relevant information in the first step, thus degrading the SSL accuracy and robustness. In the present invention, direct, one-step, approaches are employed. Namely, a one-step TDOA SSL approach and a steered beam (SB) SSL approach are employed. Each of these approaches provides an accuracy and robustness not available with the traditional two-step approaches.

摘要翻译： 提出了使用具有减轻相关和混响噪声的影响的加权因子的直接方法来发现声源的位置的系统和过程。 当使用两个以上的麦克风时，传统的基于时间延迟（TDOA）的声源定位（SSL）方法涉及两个步骤。 第一步计算每个麦克风对的TDOA，第二步合并这些估计。 这两步过程在第一步中丢弃相关信息，从而降低了SSL的准确性和鲁棒性。 在本发明中，采用直接的一步法。 也就是说，采用一步式TDOA SSL方法和转向束（SB）SSL方法。 这些方法中的每一种提供了传统的两步方法不可用的精度和鲁棒性。

公开(公告)号：US07151843B2

公开(公告)日：2006-12-19

申请号：US11042015

申请日：2005-01-25

申请人： Yong Rui ,  Yunqiang Chen

发明人： Yong Rui ,  Yunqiang Chen

IPC分类号： G06K9/00 ,  H04N7/14 ,  H04N5/225

CPC分类号： G06K9/00234 ,  G06T7/251 ,  G06T2207/10016 ,  G06T2207/30196 ,  G06T2207/30201

摘要： Automatic detection and tracking of multiple individuals includes receiving a frame of video and/or audio content and identifying a candidate area for a new face region in the frame. One or more hierarchical verification levels are used to verify whether a human face is in the candidate area, and an indication made that the candidate area includes a face if the one or more hierarchical verification levels verify that a human face is in the candidate area. A plurality of audio and/or video cues are used to track each verified face in the video content from frame to frame.

公开(公告)号：US20060227977A1

公开(公告)日：2006-10-12

申请号：US11190241

申请日：2005-07-26

申请人： Yong Rui ,  Dinei Florencio

发明人： Yong Rui ,  Dinei Florencio

IPC分类号： H04R5/00

CPC分类号： H04R3/005 ,  G10L21/0272 ,  G10L2021/02165

摘要： A system and process for finding the location of a sound source using direct approaches having weighting factors that mitigate the effect of both correlated and reverberation noise is presented. When more than two microphones are used, the traditional time-delay-of-arrival (TDOA) based sound source localization (SSL) approach involves two steps. The first step computes TDOA for each microphone pair, and the second step combines these estimates. This two-step process discards relevant information in the first step, thus degrading the SSL accuracy and robustness. In the present invention, direct, one-step, approaches are employed. Namely, a one-step TDOA SSL approach and a steered beam (SB) SSL approach are employed. Each of these approaches provides an accuracy and robustness not available with the traditional two-step approaches.

公开(公告)号：US06999599B2

公开(公告)日：2006-02-14

申请号：US10164947

申请日：2002-06-07

申请人： Yong Rui ,  Yunqiang Chen

发明人： Yong Rui ,  Yunqiang Chen

IPC分类号： G06K9/00

CPC分类号： G06K9/00234 ,  G06K9/3216 ,  G06K9/6207 ,  G06T7/251 ,  G06T7/277 ,  G06T2207/10016 ,  G06T2207/30201

摘要： A system and method for object tracking using probabilistic mode-based multi-hypothesis tracking (MHT) provides for robust and computationally efficient tracking of moving objects such as heads and faces in complex environments. A mode-based multi-hypothesis tracker uses modes that are local maximums which are refined from initial samples in a parametric state space. Because the modes are highly representative, the mode-based multi-hypothesis tracker effectively models non-linear probabilistic distributions using a small number of hypotheses. Real-time tracking performance is achieved by using a parametric causal contour model to refine initial contours to nearby modes. In addition, one common drawback of conventional MHT schemes, i.e., producing only maximum likelihood estimates instead of a desired posterior probability distribution, is addressed by introducing an importance sampling framework into MHT, and estimating the posterior probability distribution from the importance function.

公开(公告)号：US20050265562A1

公开(公告)日：2005-12-01

申请号：US11182142

申请日：2005-07-15

申请人： Yong Rui

发明人： Yong Rui

IPC分类号： G10L25/93 ,  H04R3/00 ,  G10L11/06

CPC分类号： H04R3/005 ,  H04R2201/401

摘要： A system and process is described for estimating the location of a speaker using signals output by a microphone array characterized by multiple pairs of audio sensors. The location of a speaker is estimated by first determining whether the signal data contains human speech components and filtering out noise attributable to stationary sources. The location of the person speaking is then estimated using a time-delay-of-arrival based SSL technique on those parts of the data determined to contain human speech components. A consensus location for the speaker is computed from the individual location estimates associated with each pair of microphone array audio sensors taking into consideration the uncertainty of each estimate. A final consensus location is also computed from the individual consensus locations computed over a prescribed number of sampling periods using a temporal filtering technique.

摘要翻译： 描述了一种系统和过程，用于使用由多对音频传感器表征的麦克风阵列输出的信号来估计扬声器的位置。 通过首先确定信号数据是否包含人类语音分量并滤除归因于固定源的噪声来估计扬声器的位置。 然后，使用基于时间延迟的SSL技术来估计说话人的位置，以确定包含人类语音组件的数据的那些部分。 考虑到每个估计的不确定性，从与每对麦克风阵列音频传感器相关联的各个位置估计计算扬声器的共识位置。 还可以使用时间滤波技术从在规定数量的采样周期上计算的单个共识位置计算最终共识位置。

公开(公告)号：US20050114079A1

公开(公告)日：2005-05-26

申请号：US10985243

申请日：2004-11-10

申请人： Yong Rui ,  Yunqiang Chen

发明人： Yong Rui ,  Yunqiang Chen

IPC分类号： G06T7/20 ,  G10L21/02 ,  G01S13/00

CPC分类号： G06T7/277 ,  G10L2021/02166

摘要： A system and process for tracking an object state over time using particle filter sensor fusion and a plurality of logical sensor modules is presented. This new fusion framework combines both the bottom-up and top-down approaches to sensor fusion to probabilistically fuse multiple sensing modalities. At the lower level, individual vision and audio trackers can be designed to generate effective proposals for the fuser. At the higher level, the fuser performs reliable tracking by verifying hypotheses over multiple likelihood models from multiple cues. Different from the traditional fusion algorithms, the present framework is a closed-loop system where the fuser and trackers coordinate their tracking information. Furthermore, to handle non-stationary situations, the present framework evaluates the performance of the individual trackers and dynamically updates their object states. A real-time speaker tracking system based on the proposed framework is feasible by fusing object contour, color and sound source location.

摘要翻译： 提出了一种使用粒子滤波器传感器融合和多个逻辑传感器模块跟踪物体状态随时间变化的系统和过程。 这种新的融合框架将自下而上和自顶向下的方法与传感器融合相结合，以概率地融合多种感测模式。 在较低级别，个人视觉和音频跟踪器可以设计用于为定影器生成有效的建议。 在较高级别，定影器通过从多个线索的多个似然模型上验证假设来执行可靠的跟踪。 与传统融合算法不同，本框架是闭环系统，其中定影器和跟踪器协调其跟踪信息。 此外，为了处理非平稳情况，本框架评估各个跟踪器的性能并动态更新其对象状态。 基于提出的框架的实时扬声器跟踪系统可以通过融合对象轮廓，颜色和声源位置来实现。