公开(公告)号：US20060031915A1

公开(公告)日：2006-02-09

申请号：US11097533

申请日：2005-03-31

申请人： Simon Winder ,  Matthew Uyttendaele ,  Charles Zitnick ,  Richard Szeliski ,  Sing Kang

发明人： Simon Winder ,  Matthew Uyttendaele ,  Charles Zitnick ,  Richard Szeliski ,  Sing Kang

IPC分类号： H04N7/16 ,  H04B1/66 ,  H04N11/04 ,  H04N11/02 ,  H04N7/12

CPC分类号： H04N21/4347 ,  H04N13/111 ,  H04N19/109 ,  H04N19/39 ,  H04N19/593 ,  H04N19/597 ,  H04N19/61 ,  H04N19/70 ,  H04N19/96 ,  H04N21/2365

摘要： A system and process for compressing and decompressing multiple video streams depicting substantially the same dynamic scene from different viewpoints that from a grid of viewpoints. Each frame in each contemporaneous set of video frames of the multiple streams is represented by at least a two layers—a main layer and a boundary layer. Compression of the main layers involves first designating one or more of these layers in each set of contemporaneous frames as keyframes. For each set of contemporaneous frames in time sequence order, the main layer of each keyframe is compressed using an inter-frame compression technique. In addition, the main layer of each non-keyframe within the frame set under consideration is compressed using a spatial prediction compression technique. Finally, the boundary layers of each frame in the current frame set are each compressed using an intra-frame compression technique. Decompression is generally the reverse of the compression process.

摘要翻译： 一种用于压缩和解压缩从视点网格的不同观点描绘基本相同的动态场景的多个视频流的系统和过程。 多个流的每个同期视频帧集合中的每个帧由至少两层（主层和边界层）表示。 主要层的压缩包括首先将每组同期帧中的这些层中的一个或多个指定为关键帧。 对于按时间顺序排列的每组同期帧，使用帧间压缩技术对每个关键帧的主层进行压缩。 另外，使用空间预测压缩技术对所考虑的帧集合内的每个非关键帧的主层进行压缩。 最后，使用帧内压缩技术对当前帧集合中每帧的边界层进行压缩。 压缩通常与压缩过程相反。

公开(公告)号：US20050285872A1

公开(公告)日：2005-12-29

申请号：US11192639

申请日：2005-07-28

申请人： Lifeng Wang ,  Sing Bing Kang ,  Richard Szeliski ,  Heung-Yeung Shum ,  Baining Guo

发明人： Lifeng Wang ,  Sing Bing Kang ,  Richard Szeliski ,  Heung-Yeung Shum ,  Baining Guo

IPC分类号： G06T15/20 ,  G09G5/00

CPC分类号： G06T15/04

摘要： A system and process for reconstructing optimal texture maps from multiple views of a scene is described. In essence, this reconstruction is based on the optimal synthesis of textures from multiple sources. This is generally accomplished using basic image processing theory to derive the correct weights for blending the multiple views. Namely, the steps of reconstructing, warping, prefiltering, and resampling are followed in order to warp reference textures to a desired location, and to compute spatially-variant weights for optimal blending. These weights take into consideration the anisotropy in the texture projection and changes in sampling frequency due to foreshortening. The weights are combined and the computation of the optimal texture is treated as a restoration problem, which involves solving a linear system of equations. This approach can be incorporated in a variety of applications, such as texturing of 3D models, analysis by synthesis methods, super-resolution techniques, and view-dependent texture mapping.

摘要翻译： 描述用于从场景的多个视图重建最佳纹理图的系统和过程。 实质上，这种重建是基于来自多个源的纹理的最佳合成。 这通常使用基本图像处理理论来实现，以导出用于混合多个视图的正确权重。 即，遵循重构，翘曲，预过滤和重采样的步骤，以便将参考纹理扭曲到期望的位置，并计算用于最佳混合的空间变体权重。 这些权重考虑到纹理投影中的各向异性和由于缩短引起的采样频率的变化。 权重相结合，最优纹理的计算被视为恢复问题，其涉及求解线性方程组。 这种方法可以并入各种应用中，例如3D模型的纹理化，通过合成方法的分析，超分辨率技术和视图相关的纹理映射。

公开(公告)号：US20050062719A1

公开(公告)日：2005-03-24

申请号：US10982631

申请日：2004-11-04

申请人： Kenneth Hinckley ,  Michael Sinclair ,  Richard Szeliski ,  Matthew Conway ,  Erik Hanson

发明人： Kenneth Hinckley ,  Michael Sinclair ,  Richard Szeliski ,  Matthew Conway ,  Erik Hanson

IPC分类号： G06F3/03 ,  G06F3/039 ,  G09G5/00 ,  G09G5/08

CPC分类号： G06F3/0317 ,  G06F3/0346 ,  G06F3/0395

摘要： A computer input device includes a camera for capturing an image of a pattern on a surface. A processor in the computer input device determines equations for lines from the image of the pattern, applies the line equations to a transform to form transformed line equations, and uses the transformed line equations to determine the height of the computer input device above the surface.

摘要翻译： 计算机输入装置包括用于捕获表面上的图案的图像的照相机。 计算机输入装置中的处理器从图案图像确定线的等式，将线方程应用于变换以形成变换线方程，并使用变换线方程来确定计算机输入装置在表面上方的高度。

公开(公告)号：US6097854A

公开(公告)日：2000-08-01

申请号：US905023

申请日：1997-08-01

申请人： Richard Szeliski ,  Heung-Yeung Shum

发明人： Richard Szeliski ,  Heung-Yeung Shum

IPC分类号： G06T3/40 ,  G06K9/36

CPC分类号： G06K9/209 ,  G06T3/4038 ,  G06K2009/2045

摘要： The system of the invention aligns a set plural overlapping images useful in constructing a mosaic by performing patch-based alignment of the set of overlapping images to produce a set of warped images, performing block adjustment of the set of warped images to produce a set of block-adjusted images, and then performing pair-wise motion-based local warping of the set of block-adjusted images.

摘要翻译： 本发明的系统通过执行重叠图像集合的基于图像的对齐来产生一组翘曲图像，对组合的翘曲图像执行块调整以产生一组 块调整后的图像，然后对块调整后的图像进行成对的基于运动的局部翘曲。

公开(公告)号：US6009190A

公开(公告)日：1999-12-28

申请号：US905099

申请日：1997-08-01

申请人： Richard Szeliski ,  Heung-Yeung Shum

发明人： Richard Szeliski ,  Heung-Yeung Shum

IPC分类号： G06T3/40 ,  G06T7/00 ,  G06T15/04 ,  G06K9/00 ,  G06K9/36

CPC分类号： G06K9/209 ,  G06T15/04 ,  G06T3/4038 ,  G06T7/0024 ,  G06K2009/2045

摘要： The texture map construction method and apparatus of the invention constructs, from a set of overlapping images, a texture map divisible into plural faces. This is accomplished for each one of the plural faces by computing a texture mapping transform which maps between pixel locations in the texture map and a three-dimensional coordinate system. For each image of the set, an image transform is computed which maps between pixel locations in the image and the three-dimensional coordinate system. For each image of the set, the texture mapping transform and the image transform are combined to produce a texture map-to-image space transform mapping between pixel locations in the image and pixel locations in the texture map. For each one of the pixel locations in the one face of the texture map, the system computes from the texture map-to-image space transform a pixel value of a pixel location in the image corresponding to the one pixel location in the one face of the texture map. This produces a set of image pixel values from the set of overlapping images for the one pixel location in the texture space. Finally, the set of image pixel values are blended to produce a composite pixel value for the one pixel location in the one face of the texture map.

摘要翻译： 本发明的纹理图构造方法和装置从一组重叠图像构造可分割成多个面的纹理图。 这通过计算在纹理映射图中的像素位置和三维坐标系之间映射的纹理映射变换来实现多个面中的每一个面。 对于集合的每个图像，计算在图像中的像素位置和三维坐标系之间映射的图像变换。 对于集合的每个图像，组合纹理映射变换和图像变换以在图像中的像素位置和纹理映射中的像素位置之间产生纹理映射到图像空间变换映射。 对于纹理映射的一个面中的每个像素位置，系统从纹理映射到图像空间中计算对应于一个像素中的一个像素位置的图像中的像素位置的像素值 纹理贴图。 这从纹理空间中的一个像素位置的重叠图像集合产生一组图像像素值。 最后，将该组图像像素值混合以产生纹理图的一个面中的一个像素位置的复合像素值。

公开(公告)号：US20120237111A1

公开(公告)日：2012-09-20

申请号：US13047799

申请日：2011-03-15

申请人： Sudipta N. Sinha ,  Richard Roberts ,  Drew Steedly ,  Richard Szeliski

发明人： Sudipta N. Sinha ,  Richard Roberts ,  Drew Steedly ,  Richard Szeliski

IPC分类号： G06K9/00

CPC分类号： G06K9/6892 ,  G06K9/469 ,  G06T7/579 ,  G06T2207/20072 ,  G06T2207/20076 ,  G06T2207/30244

摘要： A technology is described for performing structure from motion for unordered images of a scene with multiple object instances. An example method can include obtaining a pairwise match graph using interest point detection for obtaining interest points in images of the scene to identify pairwise image matches using the interest points. Multiple metric two-view and three-view partial reconstructions can be estimated by performing independent structure from motion computation on a plurality of match-pairs and match-triplets selected from the pairwise match graph. Pairwise image matches can be classified into correct matches and erroneous matches using expectation maximization to generate geometrically consistent match labeling hypotheses and a scoring function to evaluate the match labeling hypotheses. A structure from motion computation can then be performed on the subset of match pairs which have been inferred as correct.

摘要翻译： 描述了用于对具有多个对象实例的场景的无序图像的运动执行结构的技术。 示例性方法可以包括使用兴趣点检测获得成对匹配图，以获得场景图像中的兴趣点，以使用兴趣点来识别成对图像匹配。 可以通过对从成对匹配图中选择的多个匹配对和匹配三元组上的运动计算执行独立结构来估计多度量双视图和三视图部分重建。 成对图像匹配可以使用期望最大化分类为正确匹配和错误匹配，以生成几何一致的匹配标签假设和评分函数来评估匹配标签假设。 然后可以对已经被推断为正确的匹配对的子集执行来自运动计算的结构。

公开(公告)号：US08139886B2

公开(公告)日：2012-03-20

申请号：US12143837

申请日：2008-06-23

申请人： Richard Szeliski ,  Neel Joshi ,  Charles Lawrence Zitnick

发明人： Richard Szeliski ,  Neel Joshi ,  Charles Lawrence Zitnick

IPC分类号： G06K9/40 ,  G06K9/48

CPC分类号： G06T5/003 ,  G06T2207/20008

摘要： A two-dimensional blur kernel is computed for a digital image by first estimating a sharp image from the digital image. The sharp image is derived from the digital image by sharpening at least portions of the digital image. The two-dimensional blur function is computed by minimizing an optimization algorithm that estimates the blur function.

摘要翻译： 通过首先从数字图像估计清晰图像，为数字图像计算二维模糊核。 通过锐化数字图像的至少一部分，从数字图像导出清晰图像。 通过最小化估计模糊函数的优化算法来计算二维模糊函数。

公开(公告)号：US07889948B2

公开(公告)日：2011-02-15

申请号：US12794757

申请日：2010-06-06

申请人： Drew Steedly ,  Richard Szeliski ,  Matthew Uyttendaele ,  Michael Cohen

发明人： Drew Steedly ,  Richard Szeliski ,  Matthew Uyttendaele ,  Michael Cohen

IPC分类号： G06K9/32 ,  G06K9/36

CPC分类号： G06T3/4038 ,  G06K9/32 ,  G06K2009/2045

摘要： An “Oblique Image Stitcher” provides a technique for constructing a photorealistic oblique view from a set of input images representing a series of partially overlapping views of a scene. The Oblique Image Stitcher first projects each input image onto a geometric proxy of the scene and renders the images from a desired viewpoint. Once the images have been projected onto the geometric proxy, the rendered images are evaluated to identify optimum seams along which the various images are to be blended. Once the optimum seams are selected, the images are remapped relative to those seams by leaving the mapping unchanged at the seams and interpolating a smooth mapping between the seams. The remapped images are then composited to construct the final mosaiced oblique view of the scene. The result is a mosaic image constructed by warping the input images in a photorealistic manner which agrees at seams between images.

摘要翻译： “倾斜图像拼接器”提供了一种用于从表示场景的一系列部分重叠视图的一组输入图像中构造逼真倾斜视图的技术。 倾斜图像拼接器首先将每个输入图像投影到场景的几何代理上，并从期望的角度呈现图像。 一旦将图像投影到几何代理上，则对所渲染的图像进行评估，以确定要混合各种图像的最佳接缝。 一旦选择最佳接缝，通过在接缝处保持不变，并在接缝之间插入平滑的映射，相对于那些接缝重新映射图像。 然后将重新映射的图像合成以构建场景的最终镶嵌斜视图。 结果是通过以照片写实的方式扭曲输入图像构成的马赛克图像，其在图像之间的接缝处一致。

公开(公告)号：US07822289B2

公开(公告)日：2010-10-26

申请号：US11459724

申请日：2006-07-25

申请人： Richard Szeliski

发明人： Richard Szeliski

IPC分类号： G06K9/36 ,  G06K9/40

CPC分类号： G06F17/13

摘要： A “Finite Element Preconditioner” provides locally adapted hierarchical basis functions for preconditioning large data optimization problems. For example, a few of the many typical graphics applications that make use of iterative optimization solutions include tone mapping, gradient domain blending, colorization, and scattered data interpolation. Preconditioning image data for such optimization problems allows those image optimization problems to be solved using less computational overhead and therefore to produce better quality image outputs with less computational overhead. The Finite Element Preconditioner evaluates data, such as image data, to adapt hierarchical basis functions to inhomogeneous problems for preconditioning large optimization problems. Specifically, the Finite Element Preconditioner evaluates the local structure of a coefficient matrix derived from the data and performs a recursive set of variable eliminations, combined with a simplification of the resulting coarse level problems to obtain bases better suited for problems with inhomogeneous (spatially varying) data, smoothness, and boundary constraints.

摘要翻译： “有限元预处理器”提供了本地适应的分层基函数，用于预处理大数据优化问题。 例如，使用迭代优化解决方案的许多典型图形应用程序中的一些包括色调映射，梯度域混合，着色和分散数据插值。 用于这种优化问题的预处理图像数据允许使用更少的计算开销来解决图像优化问题，并且因此以较少的计算开销产生更好质量的图像输出。 有限元预处理器评估数据，如图像数据，以使分层基函数适应不均匀问题，用于预处理大型优化问题。 具体来说，有限元预处理器评估从数据导出的系数矩阵的局部结构，并执行递归的可变消除集合，结合简化所得到的粗略问题，以获得更适合于不均匀（空间变化） 数据，平滑度和边界约束。

公开(公告)号：US07583288B2

公开(公告)日：2009-09-01

申请号：US10869020

申请日：2004-06-15

申请人： Matthew Uyttendaele ,  Richard Szeliski

发明人： Matthew Uyttendaele ,  Richard Szeliski

IPC分类号： H04N7/00

CPC分类号： H04N5/23238 ,  G03B37/04 ,  G06T7/30 ,  G06T7/80 ,  G06T2200/32 ,  H04N17/002

摘要： A system and process for generating a panoramic video. Essentially, the panoramic video is created by first acquiring multiple videos of the scene being depicted. Preferably, these videos collectively depict a full 360 degree view of the surrounding scene and are captured using a multiple camera rig. The acquisition phase also includes a calibration procedure that provides information about the camera rig used to capture the videos that is used in the next phase for creating the panoramic video. This next phase, which is referred to as the authoring phase, involves mosaicing or stitching individual frames of the videos, which were captured at approximately the same moment in time, to form each frame of the panoramic video. A series of texture maps are then constructed for each frame of the panoramic video. Each texture map coincides with a portion of a prescribed environment model of the scene. The texture map representations of each frame of the panoramic video are encoded so as to facilitate their transfer and viewing. This can include compressing the panoramic video frames Such a procedure is useful in applications where the panoramic video is to be transferred over a network, such as the Internet.

摘要翻译： 用于生成全景视频的系统和过程。 本质上，通过首先获取所描绘的场景的多个视频来创建全景视频。 优选地，这些视频共同地描绘了周围场景的完整360度视图，并且使用多个相机钻机来捕获。 采集阶段还包括一个校准程序，该程序提供有关用于捕获下一阶段中用于创建全景视频的视频的摄像机的信息。 这个下一个阶段（被称为创作阶段）涉及拼接或缝合在大约相同的时刻被捕获的视频的各个帧，以形成全景视频的每个帧。 然后为全景视频的每个帧构建一系列纹理贴图。 每个纹理贴图与场景的规定环境模型的一部分重合。 对全景视频的每个帧的纹理映射表示进行编码，以便于它们的传送和观看。 这可以包括压缩全景视频帧。这样的过程在通过诸如因特网的网络传输全景视频的应用中是有用的。