公开(公告)号：US20050027515A1

公开(公告)日：2005-02-03

申请号：US10629278

申请日：2003-07-29

申请人： Xuedong Huang ,  Zicheng Liu ,  Zhengyou Zhang ,  Michael Sinclair ,  Alejandro Acero

发明人： Xuedong Huang ,  Zicheng Liu ,  Zhengyou Zhang ,  Michael Sinclair ,  Alejandro Acero

IPC分类号： G10L11/02 ,  G10L15/20 ,  G10L15/24 ,  H04R1/10 ,  H04R1/14 ,  H04R25/00 ,  G10L11/06

CPC分类号： H04R1/10 ,  G10L15/20 ,  G10L15/24 ,  G10L25/78 ,  H04R1/14 ,  H04R25/606

摘要： The present invention combines a conventional audio microphone with an additional speech sensor that provides a speech sensor signal based on an input. The speech sensor signal is generated based on an action undertaken by a speaker during speech, such as facial movement, bone vibration, throat vibration, throat impedance changes, etc. A speech detector component receives an input from the speech sensor and outputs a speech detection signal indicative of whether a user is speaking. The speech detector generates the speech detection signal based on the microphone signal and the speech sensor signal.

摘要翻译： 本发明将常规音频麦克风与基于输入提供语音传感器信号的附加话音传感器组合。 语音传感器信号基于语音中的扬声器在诸如面部运动，骨骼振动，喉部振动，喉部阻抗变化等中的动作而产生。语音检测器组件从语音传感器接收输入并输出语音检测 指示用户是否正在说话的信号。 语音检测器基于麦克风信号和语音传感器信号产生语音检测信号。

公开(公告)号：US06771810B1

公开(公告)日：2004-08-03

申请号：US09594806

申请日：2000-06-16

申请人： Zhengyou Zhang ,  Charles T. Loop

发明人： Zhengyou Zhang ,  Charles T. Loop

IPC分类号： G06K900

CPC分类号： G06K9/209 ,  G06T7/593

摘要： The present invention is embodied in a system and method for estimating epipolar geometry, in terms of a fundamental matrix, between multiple images of an object for stereo vision processing. The fundamental matrix embodies the epipolar geometry between the images. In general, the present invention includes a method for estimating epipolar geometry between multiple images of an original space given an initial estimate of the fundamental matrix found using a standard linear estimation technique. Namely, the system and method of the present invention estimates the fundamental matrix by transforming image points of multiple images into projective space. After this transformation is performed, nonlinear optimization is used with one parameterization of the fundamental matrix. The images are then inverse transformed back to the original space with a final estimate of the fundamental matrix. In addition, the original noise information is preserved during the optimization in the projective space.

摘要翻译： 本发明体现在用于基于立体视觉处理的对象的多个图像之间的基本矩阵估计对极几何的系统和方法。 基本矩阵体现了图像之间的对极几何。 通常，本发明包括一种用于估计原始空间的多个图像之间的对极几何的方法，给出使用标准线性估计技术发现的基本矩阵的初始估计。 即，本发明的系统和方法通过将多个图像的图像点变换为投影空间来估计基本矩阵。 在进行这种变换之后，使用非线性优化与基本矩阵的一个参数化。 然后将图像用基本矩阵的最终估计值反向变换回原始空间。 另外，在投影空间优化过程中，保留了原始噪声信息。

公开(公告)号：US06665440B1

公开(公告)日：2003-12-16

申请号：US09583062

申请日：2000-05-30

申请人： Zhengyou Zhang ,  Ying Shan

发明人： Zhengyou Zhang ,  Ying Shan

IPC分类号： G06K962

CPC分类号： G06K9/6211 ,  G06K9/32 ,  G06T7/593

摘要： The present invention is embodied in a curve matching system and method that is guided by a set of matched corners. The corner guided curve matching produces a geometrical representation of the scene from the images, which can be used for any suitable application, such as computer and stereo vision applications. In general, first, multiple images depicting a scene are digitally received by the system. The images are graphical images digitally received and processed. For example, the images can be two dimensional image data, such as bitmap or raster image data. Curves of the images are then matched to correlate the two images of the scene for creating three dimensional (3D) curve information, such as 3D vector or mathematical information, of the scene. This 3D vector information can then be used in any suitable manner, for example, to digitally reconstruct the scene for stereo vision applications.

摘要翻译： 本发明以曲线匹配系统和方法体现，该系统和方法由一组匹配的角指导。 角引导曲线匹配产生了来自图像的场景的几何表示，其可以用于任何合适的应用，诸如计算机和立体视觉应用。 通常，首先，描绘场景的多个图像被系统数字地接收。 图像是数字接收和处理的图形图像。 例如，图像可以是二维图像数据，例如位图或光栅图像数据。 然后匹配图像的曲线以将场景的两个图像相关联，用于创建场景的三维（3D）曲线信息，例如3D矢量或数学信息。 然后可以以任何合适的方式使用该3D矢量信息，例如，数字重建用于立体视觉应用的场景。

公开(公告)号：US06661913B1

公开(公告)日：2003-12-09

申请号：US09336218

申请日：1999-06-19

申请人： Zhengyou Zhang ,  Padmanabhan Anandan ,  Heung-Yeung Shum

发明人： Zhengyou Zhang ,  Padmanabhan Anandan ,  Heung-Yeung Shum

IPC分类号： G06K900

CPC分类号： G06K9/209 ,  G06K2209/40 ,  G06T7/246 ,  G06T7/579

摘要： The present invention is embodied in systems and methods for determining structure and motion of a three-dimensional (3D) object using two-dimensional (2D) images of the object obtained from multiple sets of views with different projection models, such as from a full perspective view and a weak perspective views. A novel fundamental matrix is derived that embodies the epipolar geometry between a full perspective view and a weak perspective view. The systems and methods of the present invention preferably uses the derived fundamental matrix together with the 2D image information of the full and weak perspective views to digitally reconstruct the 3D object and produce results with multi-resolution processing techniques. These techniques include recovering and refining motion parameters and recovering and refining structure parameters of the fundamental matrix. The results can include, for example, 3D positions of points, camera position between different views, texture maps, and the like.

摘要翻译： 本发明体现在使用具有不同投影模型的多组视图获得的对象的二维（2D）图像来确定三维（3D）对象的结构和运动的系统和方法中，例如从完整的 透视和弱视角。 导出了一种新颖的基本矩阵，其体现了全透视图和弱透视图之间的对极几何。 本发明的系统和方法优选地将衍生的基本矩阵与全部和弱透视图的2D图像信息一起数字重建3D对象并且使用多分辨率处理技术产生结果。 这些技术包括恢复和精炼运动参数，并恢复和完善基本矩阵的结构参数。 结果可以包括例如点的3D位置，不同视图之间的相机位置，纹理贴图等。

公开(公告)号：US09536312B2

公开(公告)日：2017-01-03

申请号：US13107986

申请日：2011-05-16

申请人： Cha Zhang ,  Wenwu Zhu ,  Zhengyou Zhang ,  Philip A. Chou

发明人： Cha Zhang ,  Wenwu Zhu ,  Zhengyou Zhang ,  Philip A. Chou

IPC分类号： G06T7/00 ,  G06K9/20

CPC分类号： G06T7/521 ,  G06K9/2036 ,  G06T7/593 ,  G06T2207/10048 ,  G06T2207/10152 ,  G06T2207/30196

摘要： A depth construction module is described that receives depth images provided by two or more depth capture units. Each depth capture unit generates its depth image using a structured light technique, that is, by projecting a pattern onto an object and receiving a captured image in response thereto. The depth construction module then identifies at least one deficient portion in at least one depth image that has been received, which may be attributed to overlapping projected patterns that impinge the object. The depth construction module then uses a multi-view reconstruction technique, such as a plane sweeping technique, to supply depth information for the deficient portion. In another mode, a multi-view reconstruction technique can be used to produce an entire depth scene based on captured images received from the depth capture units, that is, without first identifying deficient portions in the depth images.

摘要翻译： 描述了深度构造模块，其接收由两个或更多个深度捕获单元提供的深度图像。 每个深度捕获单元使用结构化的光技术生成其深度图像，即通过将图案投射到对象上并响应于此而接收捕获的图像。 然后，深度构造模块识别已经接收到的至少一个深度图像中的至少一个缺陷部分，其可归因于撞击物体的重叠投影图案。 然后，深度构建模块使用诸如平面扫描技术的多视图重建技术来为缺陷部分提供深度信息。 在另一种模式中，可以使用多视图重建技术来基于从深度捕获单元接收到的捕获图像来生成整个深度场景，即，没有首先识别深度图像中的不足部分。

公开(公告)号：US09270974B2

公开(公告)日：2016-02-23

申请号：US13178494

申请日：2011-07-08

申请人： Cha Zhang ,  Zhengyou Zhang

发明人： Cha Zhang ,  Zhengyou Zhang

IPC分类号： H04N13/02 ,  G06T7/00 ,  H04N13/00

CPC分类号： H04N13/246 ,  G06T7/85 ,  H04N13/20 ,  H04N13/207 ,  H04N13/25 ,  H04N13/257 ,  H04N13/271

摘要： A system described herein includes a receiver component that receives a first digital image from a color camera, wherein the first digital image comprises a planar object, and a second digital image from a depth sensor, wherein the second digital image comprises the planar object. The system also includes a calibrator component that jointly calibrates the color camera and the depth sensor based at least in part upon the first digital image and the second digital image.

摘要翻译： 本文描述的系统包括从彩色摄像机接收第一数字图像的接收器组件，其中第一数字图像包括平面物体和来自深度传感器的第二数字图像，其中第二数字图像包括平面物体。 该系统还包括校准器组件，其至少部分地基于第一数字图像和第二数字图像来共同校准彩色摄像机和深度传感器。

公开(公告)号：US09123144B2

公开(公告)日：2015-09-01

申请号：US13295009

申请日：2011-11-11

申请人： Nikolay Smolyanskiy ,  Christian F. Huitema ,  Cha Zhang ,  Lin Liang ,  Sean Eron Anderson ,  Zhengyou Zhang

发明人： Nikolay Smolyanskiy ,  Christian F. Huitema ,  Cha Zhang ,  Lin Liang ,  Sean Eron Anderson ,  Zhengyou Zhang

IPC分类号： G06K9/00 ,  G06T17/00 ,  G06T7/00

CPC分类号： G06T7/20 ,  G06K9/00 ,  G06T7/579 ,  G06T13/20 ,  G06T17/00 ,  G06T2200/08 ,  G06T2207/10012 ,  G06T2207/10016 ,  G06T2207/30201

摘要： A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

摘要翻译： 一种用于从二维面部特征点计算三维人头形状参数的三维形状参数计算系统和方法。 捕获包含用户脸部的一系列图像。 系统和方法的实施例通过随时间和各种头部姿势和面部表情检查用户的一系列捕获图像，然后计算平均值来推导用户头部的3D参数。 在包含从捕获图像获得的2D面部特征点的一批帧上构建能量函数，并且将能量函数最小化以解决对于批次帧有效的头部形状参数。 头部姿势参数和面部表情和动画参数可以在批次帧中的每个拍摄图像上变化。 在一些实施例中，使用使用单次迭代的修正的高斯 - 牛顿最小化技术来执行该最小化。

公开(公告)号：US08639042B2

公开(公告)日：2014-01-28

申请号：US12820143

申请日：2010-06-22

申请人： Zicheng Liu ,  Yingli Tian ,  Liangliang Cao ,  Zhengyou Zhang

发明人： Zicheng Liu ,  Yingli Tian ,  Liangliang Cao ,  Zhengyou Zhang

IPC分类号： G06K9/62

CPC分类号： G06K9/4642 ,  G06K9/6277 ,  G06T7/246 ,  G06T2207/30196

摘要： Described is a hierarchical filtered motion field technology such as for use in recognizing actions in videos with crowded backgrounds. Interest points are detected, e.g., as 2D Harris corners with recent motion, e.g. locations with high intensities in a motion history image (MHI). A global spatial motion smoothing filter is applied to the gradients of MHI to eliminate low intensity corners that are likely isolated, unreliable or noisy motions. At each remaining interest point, a local motion field filter is applied to the smoothed gradients by computing a structure proximity between sets of pixels in the local region and the interest point. The motion at a pixel/pixel set is enhanced or weakened based on its structure proximity with the interest point (nearer pixels are enhanced).

摘要翻译： 描述了一种分层过滤的运动场技术，例如用于识别具有拥挤背景的视频中的动作。 检测到兴趣点，例如，作为具有最近运动的2D哈里斯角，例如， 在运动历史图像（MHI）中具有高强度的位置。 将全局空间运动平滑滤波器应用于MHI的梯度以消除可能是孤立的，不可靠的或噪声运动的低强度拐角。 在每个剩余的兴趣点处，通过计算局部区域中的像素集合和兴趣点之间的结构接近度，将局部运动场滤波器应用于平滑的梯度。 基于其与兴趣点的结构接近（更近的像素被增强），像素/像素集合处的运动被增强或削弱。

公开(公告)号：US20130121526A1

公开(公告)日：2013-05-16

申请号：US13295009

申请日：2011-11-11

申请人： Nikolay Smolyanskiy ,  Christian F. Huitema ,  Cha Zhang ,  Lin Liang ,  Sean Eron Anderson ,  Zhengyou Zhang

发明人： Nikolay Smolyanskiy ,  Christian F. Huitema ,  Cha Zhang ,  Lin Liang ,  Sean Eron Anderson ,  Zhengyou Zhang

IPC分类号： G06K9/00

CPC分类号： G06T7/20 ,  G06K9/00 ,  G06T7/579 ,  G06T13/20 ,  G06T17/00 ,  G06T2200/08 ,  G06T2207/10012 ,  G06T2207/10016 ,  G06T2207/30201

摘要： A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

公开(公告)号：US08396247B2

公开(公告)日：2013-03-12

申请号：US12183078

申请日：2008-07-31

申请人： Zhengyou Zhang ,  Wanqing Li ,  Zicheng Liu

发明人： Zhengyou Zhang ,  Wanqing Li ,  Zicheng Liu

IPC分类号： G06K9/00

CPC分类号： G06K9/6892 ,  G06K9/00362 ,  G06K9/00711

摘要： A system that facilitates automatically determining an action of an animate object is described herein. The system includes a receiver component that receives video data that includes images of an animate object. The system additionally includes a determiner component that accesses a data store that includes an action graph and automatically determines an action undertaken by the animate object in the received video data based at least in part upon the action graph. The action graph comprises a plurality of nodes that are representative of multiple possible postures of the animate object. At least one node in the action graph is shared amongst multiple actions represented in the action graph.

摘要翻译： 这里描述了便于自动确定动画对象的动作的系统。 该系统包括接收包括动画对象的图像的视频数据的接收器组件。 该系统还包括确定器组件，其访问包括动作图的数据存储，并至少部分地基于动作图，自动地确定由所接收的视频数据中的动画对象所执行的动作。 动作图包括代表动画对象的多个可能姿势的多个节点。 动作图中至少有一个节点在动作图中表示的多个动作中共享。