公开(公告)号：US06668080B1

公开(公告)日：2003-12-23

申请号：US09399897

申请日：1999-09-21

申请人： Philip H. S. Torr ,  Padmananbhan Anandan ,  Richard S. Szeliski

发明人： Philip H. S. Torr ,  Padmananbhan Anandan ,  Richard S. Szeliski

IPC分类号： G06K934

CPC分类号： G06T17/10 ,  G06T7/55 ,  G06T2200/08

摘要： Automated layer extraction from 2D images making up a 3D scene, and automated image pixel assignment to layers, to provide for scene modeling, is disclosed. In one embodiment, a computer-implemented method determines a number of planes, or layers, and assigns pixels to the planes. The method can determine the number of planes by first determining the high-entropy pixels of the images, and then determining a 1-plane through a predetermined n-plane estimation, such as via a robust estimation, and a most likely x-plane estimation, where x is between 1 and n, such as via a Bayesian approach. Furthermore, the method can assign pixels via an iterative EM approach based on classifying criteria.

摘要翻译： 公开了从组成3D场景的2D图像中自动层提取，以及自动图像像素分配给图层，以提供场景建模。 在一个实施例中，计算机实现的方法确定多个平面或层，并将像素分配给平面。 该方法可以通过首先确定图像的高熵像素来确定平面的数量，然后通过预定的n平面估计（诸如通过鲁棒估计）和最可能的x平面估计来确定1平面 ，其中x在1和n之间，例如通过贝叶斯方法。 此外，该方法可以通过基于分类标准的迭代EM方法分配像素。

公开(公告)号：US06348918B1

公开(公告)日：2002-02-19

申请号：US09045519

申请日：1998-03-20

申请人： Richard S. Szeliski ,  Padmananbhan Anandan ,  Simon Baker

发明人： Richard S. Szeliski ,  Padmananbhan Anandan ,  Simon Baker

IPC分类号： G06T1500

CPC分类号： G06T7/593 ,  G06T2207/10012 ,  H04N2013/0081

摘要： A system and method for extracting structure from stereo that represents the scene as a collection of planar layers. Each layer optimally has an explicit 3D plane equation, a colored image with per-pixel opacity, and a per-pixel depth value relative to the plane. Initial estimates of the layers are recovered using techniques from parametric motion estimation. The combination of a global model (the plane) with a local correction to it (the per-pixel relative depth value) imposes enough local consistency to allow the recovery of shape in both textured and untextured regions.

摘要翻译： 一种用于从立体声中提取结构的系统和方法，其将场景表示为平面层的集合。 每个层最佳地具有显式的3D平面方程，具有每像素不透明度的彩色图像和相对于该平面的每像素深度值。 使用参数运动估计的技术来恢复层的初始估计。 全局模型（平面）与其局部校正（每像素相对深度值）的组合施加足够的局部一致性，以允许纹理和非纹理区域中的形状恢复。

公开(公告)号：US06320978B1

公开(公告)日：2001-11-20

申请号：US09045503

申请日：1998-03-20

申请人： Richard S. Szeliski ,  Padmananbhan Anandan ,  Simon Baker

发明人： Richard S. Szeliski ,  Padmananbhan Anandan ,  Simon Baker

IPC分类号： G06K900

CPC分类号： G06T7/593 ,  G06T2207/10012 ,  H04N13/239 ,  H04N2013/0081

摘要： A system and method for extracting structure from stereo that represents the scene as a collection of planar layers. Each layer optimally has an explicit 3D plane equation, a colored image with per-pixel opacity, and a per-pixel depth value relative to the plane. Initial estimates of the layers are made and then refined using a re-synthesis step which takes into account both occlusions and mixed pixels. Reasoning about these effects allows the recovery of depth and color information with high accuracy, even in partially occluded regions. Moreover, the combination of a global model (the plane) with a local correction to it (the per-pixel relative depth value) imposes enough local consistency to allow the recovery of shape in both textured and untextured regions.

摘要翻译： 一种用于从立体声中提取结构的系统和方法，其将场景表示为平面层的集合。 每个层最佳地具有显式的3D平面方程，具有每像素不透明度的彩色图像和相对于该平面的每像素深度值。 使用重新合成步骤对层进行初步估算，然后再次考虑两个遮挡和混合像素。 推论这些效果可以高精度地恢复深度和颜色信息，即使在部分遮挡的区域。 此外，全局模型（平面）与其局部校正（每像素相对深度值）的组合施加足够的局部一致性，以允许纹理和非纹理区域中的形状恢复。

公开(公告)号：US06198852B1

公开(公告)日：2001-03-06

申请号：US09088543

申请日：1998-06-01

申请人： Padmananbhan Anandan ,  Michal Irani ,  Daphna Weinshall

发明人： Padmananbhan Anandan ,  Michal Irani ,  Daphna Weinshall

IPC分类号： G06K936

CPC分类号： G06K9/209 ,  G06T7/97 ,  G06T2207/20088 ,  G06T2207/20228

摘要： The invention is embodied in a process for synthesizing a new image representing a new viewpoint of a scene from at least two existing images of the scene taken from different respective viewspoints. The process begins by choosing a planar surface visible in the at least two of the existing images and transforming the at least two existing images relative to one another so as to bring the planar surface into perspective alignment in the at least two existing images, and then choosing a reference frame and computing parallax vectors between the two images of the projection of common scene points on the reference frame. Preferably, the reference frame comprises an image plane of a first one of the existing images. Preferably, the reference frame is co-planar with the planar surface. In this case, the transforming of the existing images is achieved by performing a projective transform on a second one of the existing images to bring its image of the planar surface into perspective alignment with the image of the planar surface in the first existing image. Preferably, the image parameter of the new view comprises information sufficient, together with the parallax vectors, to deduce: (a) a trifocal ratio in the reference frame and (b) one epipole between the new viewpoint and one of the first and second viewpoints.

摘要翻译： 本发明体现在从不同的各个视点拍摄的场景的至少两个现有图像中合成表示场景的新视点的新图像的处理。 该过程开始于在至少两个现有图像中选择可见的平面表面，并相对于彼此变换至少两个现有图像，以便使平面在至少两个现有图像中进行透视对准，然后 选择参考帧并计算参考帧上的共同场景点的投影的两个图像之间的视差矢量。优选地，参考帧包括第一个现有图像的图像平面。 优选地，参考框架与平面表面共面。 在这种情况下，现有图像的变换是通过在第二个现有图像上进行投射变换来实现的，以使其平面的图像与第一个现有图像中的平面的图像进行透视对齐。 ，新视图的图像参数包括与视差矢量一起足够的信息，以推断：（a）参考帧中的三焦比和（b）新视点与第一视点和第二视点中的一个之间的一个epipole。

公开(公告)号：US06614429B1

公开(公告)日：2003-09-02

申请号：US09336550

申请日：1999-06-19

申请人： Zhengyou Zhang ,  Padmananbhan Anandan ,  Heung-Yeung Shum

发明人： Zhengyou Zhang ,  Padmananbhan Anandan ,  Heung-Yeung Shum

IPC分类号： G06T1700

CPC分类号： G06K9/209 ,  G06K2209/40 ,  G06T7/246 ,  G06T7/579 ,  G06T7/97 ,  G06T2207/10016

摘要： The present invention is embodied in systems and methods for determining structure and motion of a three-dimensional (3D) object using two-dimensional (2D) images of the object obtained from multiple sets of views with different projection models, such as from a full perspective view and a weak perspective views. A novel fundamental matrix is derived that embodies the epipolar geometry between a full perspective view and a weak perspective view. The systems and methods of the present invention preferably uses the derived fundamental matrix together with the 2D image information of the full and weak perspective views to digitally reconstruct the 3D object and produce results with multi-resolution processing techniques. These techniques include recovering and refining motion parameters and recovering and refining structure parameters of the fundamental matrix. The results can include, for example, 3D positions of points, camera position between different views, texture maps, and the like.

摘要翻译： 本发明体现在使用具有不同投影模型的多组视图获得的对象的二维（2D）图像来确定三维（3D）对象的结构和运动的系统和方法中，例如从完整的 透视和弱视角。 导出了一种新颖的基本矩阵，其体现了全透视图和弱透视图之间的对极几何。 本发明的系统和方法优选地将衍生的基本矩阵与全部和弱透视图的2D图像信息一起数字重建3D对象并且使用多分辨率处理技术产生结果。 这些技术包括恢复和精炼运动参数，并恢复和完善基本矩阵的结构参数。 结果可以包括例如点的3D位置，不同视图之间的相机位置，纹理贴图等。