公开(公告)号：US20060028473A1

公开(公告)日：2006-02-09

申请号：US10910088

申请日：2004-08-03

申请人： Matthew Uyttendaele ,  Simon Winder ,  Charles Zitnick ,  Richard Szeliski ,  Sing Kang

发明人： Matthew Uyttendaele ,  Simon Winder ,  Charles Zitnick ,  Richard Szeliski ,  Sing Kang

IPC分类号： G06T15/70 ,  G06T13/00

CPC分类号： G06T15/205

摘要： A system and process for rendering and displaying an interactive viewpoint video is presented in which a user can watch a dynamic scene while manipulating (freezing, slowing down, or reversing) time and changing the viewpoint at will. The ability to interactively control viewpoint while watching a video is an exciting new application for image-based rendering. Because any intermediate view can be synthesized at any time, with the potential for space-time manipulation, this type of video has been dubbed interactive viewpoint video.

摘要翻译： 呈现用于呈现和显示交互式视点视频的系统和过程，其中用户可以在操纵（冻结，减速或反转）时间并随意改变视点的同时观看动态场景。 在观看视频时交互控制视点的能力是基于图像的渲染的令人兴奋的新应用。 因为任何中间视图可以随时被合成，具有时空操纵的潜力，这种类型的视频被称为交互视点视频。

公开(公告)号：US20050285874A1

公开(公告)日：2005-12-29

申请号：US10879235

申请日：2004-06-28

申请人： Charles Zitnick ,  Richard Szeliski ,  Sing Kang ,  Matthew Uyttendaele ,  Simon Winder

发明人： Charles Zitnick ,  Richard Szeliski ,  Sing Kang ,  Matthew Uyttendaele ,  Simon Winder

IPC分类号： G06T13/00 ,  G06T15/10 ,  G06T15/70 ,  G09G5/02

CPC分类号： G06T15/205

摘要： A system and process for generating a two-layer, 3D representation of a digital or digitized image from the image and a pixel disparity map of the image is presented. The two layer representation includes a main layer having pixels exhibiting background colors and background disparities associated with correspondingly located pixels of depth discontinuity areas in the image, as well as pixels exhibiting colors and disparities associated with correspondingly located pixels of the image not found in these depth discontinuity areas. The other layer is a boundary layer made up of pixels exhibiting foreground colors, foreground disparities and alpha values associated with the correspondingly located pixels of the depth discontinuity areas. The depth discontinuity areas correspond to prescribed sized areas surrounding depth discontinuities found in the image using a disparity map thereof.

摘要翻译： 提出了一种用于从图像生成数字或数字化图像的二层3D表示和图像的像素视差图的系统和过程。 两层表示包括具有显示背景颜色的像素和与图像中的深度不连续区域的相应定位的像素相关联的背景差异的主层以及与在这些深度中未找到的图像的相应定位的像素相关联的颜色和差异的像素 不连续区域。 另一层是由与前述深度不连续区域的对应位置的像素相关联的前景色，前景差异和α值的像素构成的边界层。 深度不连续区域对应于使用其视差图在图像中发现的围绕深度不连续性的规定尺寸的区域。

公开(公告)号：US20050093877A1

公开(公告)日：2005-05-05

申请号：US10983193

申请日：2004-11-05

申请人： Lifeng Wang ,  Sing Kang ,  Richard Szeliski ,  Heung-Yeung Shum ,  Baining Guo

发明人： Lifeng Wang ,  Sing Kang ,  Richard Szeliski ,  Heung-Yeung Shum ,  Baining Guo

IPC分类号： G09G5/00

CPC分类号： G06T11/001

摘要： A system and process for reconstructing optimal texture maps from multiple views of a scene is described. In essence, this reconstruction is based on the optimal synthesis of textures from multiple sources. This is generally accomplished using basic image processing theory to derive the correct weights for blending the multiple views. Namely, the steps of reconstructing, warping, prefiltering, and resampling are followed in order to warp reference textures to a desired location, and to compute spatially-variant weights for optimal blending. These weights take into consideration the anisotropy in the texture projection and changes in sampling frequency due to foreshortening. The weights are combined and the computation of the optimal texture is treated as a restoration problem, which involves solving a linear system of equations. This approach can be incorporated in a variety of applications, such as texturing of 3D models, analysis by synthesis methods, super-resolution techniques, and view-dependent texture mapping.

摘要翻译： 描述用于从场景的多个视图重建最佳纹理图的系统和过程。 实质上，这种重建是基于来自多个源的纹理的最佳合成。 这通常使用基本图像处理理论来实现，以导出用于混合多个视图的正确权重。 即，遵循重构，翘曲，预过滤和重采样的步骤，以便将参考纹理扭曲到期望的位置，并计算用于最佳混合的空间变体权重。 这些权重考虑到纹理投影中的各向异性和由于缩短引起的采样频率的变化。 权重相结合，最优纹理的计算被视为恢复问题，其涉及求解线性方程组。 这种方法可以并入各种应用中，例如3D模型的纹理化，通过合成方法的分析，超分辨率技术和视图相关的纹理映射。

公开(公告)号：US06879731B2

公开(公告)日：2005-04-12

申请号：US10425338

申请日：2003-04-29

申请人： Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder ,  Richard Szeliski

发明人： Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder ,  Richard Szeliski

IPC分类号： H04N5/235 ,  H04N5/77 ,  H04N5/781 ,  H04N5/85 ,  G06K9/40

CPC分类号： H04N5/2355 ,  G06T5/50 ,  H04N5/235 ,  H04N5/2352 ,  H04N5/77 ,  H04N5/781 ,  H04N5/85

摘要： A system and process for generating High Dynamic Range (HDR) video is presented which involves first capturing a video image sequence while varying the exposure so as to alternate between frames having a shorter and longer exposure. The exposure for each frame is set prior to it being captured as a function of the pixel brightness distribution in preceding frames. Next, for each frame of the video, the corresponding pixels between the frame under consideration and both preceding and subsequent frames are identified. For each corresponding pixel set, at least one pixel is identified as representing a trustworthy pixel. The pixel color information associated with the trustworthy pixels is then employed to compute a radiance value for each pixel set to form a radiance map. A tone mapping procedure can then be performed to convert the radiance map into an 8-bit representation of the HDR frame.

摘要翻译： 提出了用于产生高动态范围（HDR）视频的系统和过程，其涉及首先在改变曝光的同时捕获视频图像序列，以便在具有较短和较长曝光的帧之间交替。 每个帧的曝光在其被捕获之前被设置为在先前帧中的像素亮度分布的函数。 接下来，对于视频的每个帧，识别所考虑的帧与前后帧之间的对应像素。 对于每个对应的像素集合，至少一个像素被识别为表示可靠的像素。 然后使用与可信赖像素相关联的像素颜色信息来计算每个像素组的辐射值以形成辐射图。 然后可以执行色调映射过程以将辐射图转换成HDR帧的8位表示。

公开(公告)号：US06825850B2

公开(公告)日：2004-11-30

申请号：US10283562

申请日：2002-10-30

申请人： Lifeng Wang ,  Sing Bing Kang ,  Richard Szeliski ,  Heung-Yeung Shum ,  Baining Guo

发明人： Lifeng Wang ,  Sing Bing Kang ,  Richard Szeliski ,  Heung-Yeung Shum ,  Baining Guo

IPC分类号： G09G500

CPC分类号： G06T11/001

摘要： A system and process for reconstructing optimal texture maps from multiple views of a scene is described. In essence, this reconstruction is based on the optimal synthesis of textures from multiple sources. This is generally accomplished using basic image processing theory to derive the correct weights for blending the multiple views. Namely, the steps of reconstructing, warping, prefiltering, and resampling are followed in order to warp reference textures to a desired location, and to compute spatially-variant weights for optimal blending. These weights take into consideration the anisotropy in the texture projection and changes in sampling frequency due to foreshortening. The weights are combined and the computation of the optimal texture is treated as a restoration problem, which involves solving a linear system of equations. This approach can be incorporated in a variety of applications, such as texturing of 3D models, analysis by synthesis methods, super-resolution techniques, and view-dependent texture mapping.

公开(公告)号：US6075545A

公开(公告)日：2000-06-13

申请号：US960510

申请日：1997-10-29

申请人： Jeremy S. De Bonet ,  Richard Szeliski

发明人： Jeremy S. De Bonet ,  Richard Szeliski

IPC分类号： G06T1/60 ,  G06F12/06

CPC分类号： G06T1/60

摘要： Methods and apparatus for storing, accessing, and processing information representing images through the use of row and column pointers are described. By manipulating and/or generating new sets of row and column pointers, many image processing operations can be performed without the need to access or copy the original image data. Padding, enlargement and reduction operations are examples of image processing operations that can be performed virtually. A logical image is created as the result of a virtual image processing operation. In order to permit the fast and efficient access of the image data which represents the logical image, the logical image is divided into safe and unsafe logical image regions. In a safe logical image region, data representing the image is regularly spaced in memory and may be accessed using a first relatively fast and efficient memory access technique. In unsafe logical image regions, the data representing the logical image is not regularly spaced in memory and is accessed using a second memory access technique that uses both the row and column pointers associated with the unsafe image region. The methods and apparatus of the present invention allow many image processing operations to be performed using less memory and/or by performing fewer computations than conventional image processing techniques.

摘要翻译： 描述了通过使用行和列指针来存储，访问和处理表示图像的信息的方法和装置。 通过操作和/或生成新的行和列指针集合，可以执行许多图像处理操作，而不需要访问或复制原始图像数据。 填充，放大和缩小操作是可以虚拟执行的图像处理操作的示例。 作为虚拟图像处理操作的结果，创建逻辑图像。 为了允许代表逻辑图像的图像数据的快速和有效的访问，逻辑图像被划分为安全和不安全的逻辑图像区域。 在安全的逻辑图像区域中，表示图像的数据在存储器中规则地间隔开，并且可以使用第一相对快速和有效的存储器访问技术来访问。 在不安全的逻辑图像区域中，表示逻辑图像的数据在存储器中不规则地间隔，并且使用使用与不安全图像区域相关联的行和列指针的第二存储器访问技术进行访问。 本发明的方法和装置允许使用较少的存储器和/或通过执行比常规图像处理技术更少的计算来执行许多图像处理操作。

公开(公告)号：US5986668A

公开(公告)日：1999-11-16

申请号：US905103

申请日：1997-08-01

申请人： Richard Szeliski ,  Heung-Yeung Shum

发明人： Richard Szeliski ,  Heung-Yeung Shum

IPC分类号： G06T3/40 ,  G06T7/00 ,  G06F15/00

CPC分类号： G06T7/0028 ,  G06T3/4038

摘要： The invention is embodied in a deghosting method and apparatus which locally aligns individual images in a set of overlapping images of a mosaic. This is accomplished by determining, at plural predetermined pixel locations of each one of the images, motions between the one image and other images of the set, combining the motions to produce an estimated motion at each of the plural predetermined pixel locations of the one image, and then warping the one image in accordance with the estimated motions. Preferably, it is first which of the images of the set overlies the one image. This determination is made by determining alignment transformations relating the images to a 3-dimensional coordinate system and then inferring mutual overlap between images from the transformations. The images are resampled in accordance with these alignment transformations. The warping of each image is accomplished by constructing a mapping of warped pixel locations from the estimated motions and then resampling the one image at the warped pixel locations. The mapping is preferably a reverse mapping of pixels in an unwarped version of the one image.

摘要翻译： 本发明体现在以马赛克的重叠图像的集合中的各个图像局部对齐的消反像方法和装置。 这是通过在每个图像的多个预定像素位置处确定一个图像与该组的其他图像之间的运动来组合运动以在一个图像的多个预定像素位置的每一个处产生估计的运动来实现的 ，然后根据估计的动作扭曲一幅图像。 优选地，集合中的哪一个图像覆盖在一个图像上。 通过确定将图像与三维坐标系相关联的对准变换确定，然后推断来自变换的图像之间的相互重叠。 根据这些对准转换重新采样图像。 每个图像的翘曲通过从估计的运动构建翘曲的像素位置的映射，然后在翘曲的像素位置重新采样一个图像来实现。 该映射优选地是该一个图像的不正确版本中的像素的反向映射。

公开(公告)号：US08358359B2

公开(公告)日：2013-01-22

申请号：US12690929

申请日：2010-01-21

申请人： Simon J. Baker ,  Eric P. Bennett ,  Sing Bing Kang ,  Richard Szeliski

发明人： Simon J. Baker ,  Eric P. Bennett ,  Sing Bing Kang ,  Richard Szeliski

IPC分类号： H04N5/217 ,  H04N11/00 ,  H04N7/00 ,  G06K9/40

CPC分类号： G06T5/003 ,  G06T2207/20201

摘要： A system is described for reducing artifacts produced by a rolling shutter capture technique in the presence of high-frequency motion, e.g., produced by large accelerations or jitter. The system operates by computing low-frequency information based on the motion of points from one frame to the next. The system then uses the low-frequency information to infer the high-frequency motion, e.g., by treating the low-frequency information as known integrals of the unknown underlying high-frequency information. The system then uses the high-frequency information to reduce the presence of artifacts. In effect, the correction aims to re-render video information as though all the pixels in each frame were imaged at the same time using a global shutter technique. An auto-calibration module can estimate the value of a capture parameter, which relates to a time interval between the capture of two subsequent rows of video information.

摘要翻译： 描述了一种用于在存在高频运动（例如由大的加速度或抖动产生的）的情况下减少由快门拍摄技术产生的假象的系统。 该系统通过基于从一帧到下一帧的点的运动来计算低频信息来进行操作。 然后，系统使用低频信息来推断高频运动，例如通过将低频信息作为未知底层高频信息的已知积分来处理。 然后，系统使用高频信息来减少伪像的存在。 实际上，校正旨在重新渲染视频信息，就像使用全局快门技术同时成像每个帧中的所有像素一样。 自动校准模块可以估计捕获参数的值，其涉及捕获两个后续行的视频信息之间的时间间隔。

公开(公告)号：US08189959B2

公开(公告)日：2012-05-29

申请号：US12104446

申请日：2008-04-17

申请人： Richard Szeliski ,  Matthew T. Uyttendaele

发明人： Richard Szeliski ,  Matthew T. Uyttendaele

IPC分类号： G06K9/36

CPC分类号： G06T3/4038

摘要： Multi-spline image blending technique embodiments are presented which generally employ a separate low-resolution offset field for every image region being blended, rather than a single (piecewise smooth) offset field for all the regions to produce a visually consistent blended image. Each of the individual offset fields is smoothly varying, and so is represented using a low-dimensional spline. A resulting linear system can be rapidly solved because it involves many fewer variables than the number of pixels being blended.

摘要翻译： 提出了多样条图像混合技术实施例，其通常对于混合的每个图像区域采用单独的低分辨率偏移场，而不是针对所有区域的单个（分段平滑）偏移场，以产生视觉上一致的混合图像。 每个单独的偏移场都是平滑变化的，因此使用低维样条表示。 由此产生的线性系统可以快速解决，因为它涉及比混合像素数少许多的变量。

公开(公告)号：US20120114037A1

公开(公告)日：2012-05-10

申请号：US13348262

申请日：2012-01-11

申请人： Simon Winder ,  Matthew T. Uyttendaele ,  Charles Lawrence Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang

发明人： Simon Winder ,  Matthew T. Uyttendaele ,  Charles Lawrence Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang

IPC分类号： H04N11/02

CPC分类号： H04N21/4347 ,  H04N13/111 ,  H04N19/109 ,  H04N19/39 ,  H04N19/593 ,  H04N19/597 ,  H04N19/61 ,  H04N19/70 ,  H04N19/96 ,  H04N21/2365

摘要： A process for compressing and decompressing non-keyframes in sequential sets of contemporaneous video frames making up multiple video streams where the video frames in a set depict substantially the same scene from different viewpoints. Each set of contemporaneous video frames has a plurality frames designated as keyframes with the remaining being non-keyframes. In one embodiment, the non-keyframes are compressed using a multi-directional spatial prediction technique. In another embodiment, the non-keyframes of each set of contemporaneous video frames are compressed using a combined chaining and spatial prediction compression technique. The spatial prediction compression technique employed can be a single direction technique where just one reference frame, and so one chain, is used to predict each non-keyframe, or it can be a multi-directional technique where two or more reference frames, and so chains, are used to predict each non-keyframe.

摘要翻译： 一种用于在构成多个视频流的同步视频帧的顺序集合中压缩和解压缩非关键帧的过程，其中集合中的视频帧从不同视点描绘基本上相同的场景。 每组同时期的视频帧具有指定为关键帧的多个帧，其余的是非关键帧。 在一个实施例中，使用多方向空间预测技术来压缩非关键帧。 在另一个实施例中，使用组合链接和空间预测压缩技术来压缩每组同时期视频帧的非关键帧。 所使用的空间预测压缩技术可以是单向技术，其中仅使用一个参考帧，因此使用一条链来预测每个非关键帧，或者它可以是多方向技术，其中两个或更多个参考帧等 链，用于预测每个非关键帧。