公开(公告)号：US20080112642A1

公开(公告)日：2008-05-15

申请号：US11559861

申请日：2006-11-14

Applicant: Yasuyuki Matsushita ,  Sing Bing Kang ,  Xiaoou Tang ,  Takaaki Shiratori

Inventor： Yasuyuki Matsushita ,  Sing Bing Kang ,  Xiaoou Tang ,  Takaaki Shiratori

IPC: G06K9/40

CPC classification number: G06T5/005 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10016 ,  G06T2207/20016

Abstract: Systems and methods for video completion by motion field transfer are described. In one aspect, a spatio-temporal target patch of an input video data sequence is filled in or replaced by motion field transfer from a spatio-temporal source patch of the input video data sequence. Color is propagated to corresponding portions of the spatio-temporal target patch by treating the transferred motion information as directed edges These motion field transfer and color propagation operations result in a video completed spatio-temporal target patch. The systems and methods present the video data sequence, which now includes the video completed spatio-temporal target patch, to user for viewing.

Abstract translation: 描述了通过运动场传输进行视频完成的系统和方法。 在一个方面，输入视频数据序列的时空目标贴片由输入视频数据序列的时空源片段填充或由运动场传输代替。 通过将转移的运动信息作为有向边缘进行处理，将颜色传播到时空目标贴片的相应部分。这些运动场转移和颜色传播操作导致视频完成的时空目标贴片。 系统和方法呈现视频数据序列，其现在包括视频完成的时空目标补丁，供用户观看。

公开(公告)号：US07292257B2

公开(公告)日：2007-11-06

申请号：US10880774

申请日：2004-06-28

Applicant: Sing Bing Kang ,  Charles Zitnick, III ,  Matthew Uyttendaele ,  Simon Winder ,  Richard Szeliski

Inventor： Sing Bing Kang ,  Charles Zitnick, III ,  Matthew Uyttendaele ,  Simon Winder ,  Richard Szeliski

IPC: G09G5/00 ,  G06T15/00 ,  G06T15/40 ,  G06K9/00

CPC classification number: H04N5/2627 ,  G06T7/557 ,  G06T7/596 ,  G06T15/205 ,  G06T2207/10021

Abstract: A system and process for generating, and then rendering and displaying, an interactive viewpoint video in which a user can watch a dynamic scene while manipulating (freezing, slowing down, or reversing) time and changing the viewpoint at will. In general, the interactive viewpoint video is generated using a small number of cameras to capture multiple video streams. A multi-view 3D reconstruction and matting technique is employed to create a layered representation of the video frames that enables both efficient compression and interactive playback of the captured dynamic scene, while at the same time allowing for real-time rendering.

Abstract translation: 一种用于生成并再现和显示交互视点视频的系统和过程，其中用户可以在操纵（冻结，减慢或反转）时间并随意改变视点的同时观看动态场景。 通常，使用少量摄像机生成交互视点视频以捕获多个视频流。 采用多视图3D重建和消隐技术来创建视频帧的分层表示，使得能够实现捕获的动态场景的高效压缩和交互式回放，同时允许实时渲染。

公开(公告)号：US07152024B2

公开(公告)日：2006-12-19

申请号：US11217540

申请日：2005-09-01

Applicant: Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

Inventor： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

IPC: G06K9/00 ,  G06F7/60

CPC classification number: G06K9/4661 ,  G06K9/00268 ,  G06T7/521 ,  G06T15/506

Abstract: In the described embodiment, methods and systems for processing facial image data for use in animation are described. In one embodiment, a system is provided that illuminates a face with illumination that is sufficient to enable the simultaneous capture of both structure data, e.g. a range or depth map, and reflectance properties, e.g. the diffuse reflectance of a subject's face. This captured information can then be used for various facial animation operations, among which are included expression recognition and expression transformation.

公开(公告)号：US07142209B2

公开(公告)日：2006-11-28

申请号：US11097549

申请日：2005-03-31

Applicant: Matthew Uyttendaele ,  Simon Winder ,  Charles Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang

Inventor： Matthew Uyttendaele ,  Simon Winder ,  Charles Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang

IPC: G06T15/20

CPC classification number: G06T15/205 ,  G09G5/14 ,  G09G2320/0261 ,  G09G2340/10

Abstract: A system and process for rendering and displaying an interactive viewpoint video is presented in which a user can watch a dynamic scene while manipulating (freezing, slowing down, or reversing) time and changing the viewpoint at will. The ability to interactively control viewpoint while watching a video is an exciting new application for image-based rendering. Because any intermediate view can be synthesized at any time, with the potential for space-time manipulation, this type of video has been dubbed interactive viewpoint video.

Abstract translation: 呈现用于呈现和显示交互式视点视频的系统和过程，其中用户可以在操纵（冻结，减速或反转）时间并随意改变视点的同时观看动态场景。 在观看视频时交互控制视点的能力是基于图像的渲染的令人兴奋的新应用。 因为任何中间视图可以随时被合成，具有时空操纵的潜力，这种类型的视频被称为交互视点视频。

公开(公告)号：US07046401B2

公开(公告)日：2006-05-16

申请号：US09872725

申请日：2001-06-01

Applicant: Frederic Dufaux ,  Sing Bing Kang ,  Robert Alan Ulichney

Inventor： Frederic Dufaux ,  Sing Bing Kang ,  Robert Alan Ulichney

IPC: H04N1/387 ,  H04N1/04 ,  G06K9/48 ,  G06K9/40 ,  G06K9/00

CPC classification number: H04N1/3876 ,  G06K9/20 ,  G06T3/0075 ,  G06T5/006 ,  G06T7/207 ,  G06T7/30 ,  H04N1/195 ,  H04N1/19594 ,  H04N2201/0414 ,  H04N2201/0436

Abstract: A camera-based document scanning system produces electronic versions of documents, based on a plurality of images of discrete portions of the documents. The system compares each pair of consecutive images and derives motion parameters that indicate the relative motion between each pair of consecutive images. The system utilizes the derived motion parameters to align and merge each image with respect to the previous images, thereby building a single, mosaic image of the document. In the illustrative embodiment, the motion parameters are derived by minimizing a sum of squared differences equation on a pixel-by-pixel basis.

公开(公告)号：US20060072851A1

公开(公告)日：2006-04-06

申请号：US11286524

申请日：2005-11-22

Applicant: Sing Bing Kang ,  Richard Szeliski ,  Matthew Uyttendaele

Inventor： Sing Bing Kang ,  Richard Szeliski ,  Matthew Uyttendaele

IPC: G06K9/32 ,  G06K9/36

CPC classification number: G06T5/20 ,  G06K9/32 ,  G06K2009/2045 ,  G06T3/00 ,  G06T5/003 ,  G06T7/30 ,  G06T7/55 ,  G06T15/503

Abstract: A system and method for deghosting mosaics provides a novel multiperspective plane sweep approach for generating an image mosaic from a sequence of still images, video images, scanned photographic images, computer generated images, etc. This multiperspective plane sweep approach uses virtual camera positions to compute depth maps for columns of overlapping pixels in adjacent images. Object distortions and ghosting caused by image parallax when generating the image mosaics are then minimized by blending pixel colors, or grey values, for each computed depth to create a common composite area for each of the overlapping images. Further, the multiperspective plane sweep approach described herein is both computationally efficient, and applicable to both the case of limited overlap between the images used for creating the image mosaics, and to the case of extensive or increased image overlap.

公开(公告)号：US07015926B2

公开(公告)日：2006-03-21

申请号：US10879235

申请日：2004-06-28

Applicant: Charles Lawrence Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder

Inventor： Charles Lawrence Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder

IPC: G09G5/02

CPC classification number: G06T15/205

Abstract: A system and process for generating a two-layer, 3D representation of a digital or digitized image from the image and a pixel disparity map of the image is presented. The two layer representation includes a main layer having pixels exhibiting background colors and background disparities associated with correspondingly located pixels of depth discontinuity areas in the image, as well as pixels exhibiting colors and disparities associated with correspondingly located pixels of the image not found in these depth discontinuity areas. The other layer is a boundary layer made up of pixels exhibiting foreground colors, foreground disparities and alpha values associated with the correspondingly located pixels of the depth discontinuity areas. The depth discontinuity areas correspond to prescribed sized areas surrounding depth discontinuities found in the image using a disparity map thereof.

Abstract translation: 提出了一种用于从图像生成数字或数字化图像的二层3D表示和图像的像素视差图的系统和过程。 两层表示包括具有显示背景颜色的像素和与图像中的深度不连续区域的相应定位的像素相关联的背景差异的主层以及与在这些深度中未找到的图像的相应定位的像素相关联的颜色和差异的像素 不连续区域。 另一层是由与前述深度不连续区域的对应位置的像素相关联的前景色，前景差异和α值的像素构成的边界层。 深度不连续区域对应于使用其视差图在图像中发现的围绕深度不连续性的规定尺寸的区域。

公开(公告)号：US06968973B2

公开(公告)日：2005-11-29

申请号：US10449700

申请日：2003-05-31

Applicant: Matthew T. Uyttendaele ,  Sing Bing Kang ,  Richard Szeliski ,  Antonio Criminisi

Inventor： Matthew T. Uyttendaele ,  Sing Bing Kang ,  Richard Szeliski ,  Antonio Criminisi

IPC: G06F17/00 ,  G06F17/30 ,  G07F11/00 ,  G09G5/00

CPC classification number: G06F17/30056 ,  G06F17/30855

Abstract: A system and process for providing an interactive video tour of a tour site to a user is presented. In general, the system and process provides an image-based rendering system that enables users to explore remote real world locations, such as a house or a garden. The present approach is based directly on filming an environment, and then using image-based rendering techniques to replay the tour in an interactive manner. As such, the resulting experience is referred to as Interactive Video Tours. The experience is interactive in that the user can move freely along a path, choose between different directions of motion at branch points in the path, and look around in any direction. The user experience is additionally enhanced with multimedia elements such as overview maps, video textures, and sound.

公开(公告)号：US06850872B1

公开(公告)日：2005-02-01

申请号：US09652600

申请日：2000-08-30

Applicant: Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

Inventor： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

IPC: G06T1/00 ,  G06F7/60 ,  G06F9/455 ,  G06F17/10 ,  G06K9/00 ,  G06K9/46 ,  G06T7/00 ,  G06T15/50 ,  G06T15/60 ,  G06F101/00

CPC classification number: G06K9/4661 ,  G06K9/00268 ,  G06T7/521 ,  G06T15/506

Abstract: In the described embodiment, methods and systems for processing facial image data for use in animation are described. In one embodiment, a system is provided that illuminates a face with illumination that is sufficient to enable the simultaneous capture of both structure data, e.g. a range or depth map, and reflectance properties, e.g. the diffuse reflectance of a subject's face. This captured information can then be used for various facial animation operations, among which are included expression recognition and expression transformation.

Abstract translation: 在所描述的实施例中，描述了用于处理用于动画的面部图像数据的方法和系统。 在一个实施例中，提供了一种系统，其利用足以使得能够同时捕获结构数据的照明来照亮面部，例如， 范围或深度图，以及反射率特性，例如。 受试者面部的漫反射。 然后，该捕获的信息可以用于各种面部动画操作，其中包括表达式识别和表达变换。

公开(公告)号：US06256058B1

公开(公告)日：2001-07-03

申请号：US08660704

申请日：1996-06-06

Applicant: Sing Bing Kang ,  Richard S. Weiss

Inventor： Sing Bing Kang ,  Richard S. Weiss

IPC: H04N5225

CPC classification number: H04N5/23238 ,  H04N5/2624

Abstract: In a computerized image processing system, a camera acquires a set of images of a scene while rotating the camera about an axis passing through an optical center of the camera. The images of the set overlap each other. An initial estimate of the focal length of the camera is made. The initial focal length can be any reasonable focal length. Using the initial estimate of the focal length, the set of images are composited in a memory to determine an estimated initial composited length. A next best estimate of the focal length is derived from the initial estimated composited length. The set of images are recomposed using the next best focal length estimate. This process is iterated until the absolute difference between the successive estimates of the focal length is less than a predetermined threshold to calibrate the camera. In addition, the process of compositing the set of images can use a weighted interpolation scheme to reduce the blurring effects caused by noise and digitization effects and incorrect focal length.

Abstract translation: 在计算机化图像处理系统中，摄像机围绕通过照相机的光学中心的轴旋转摄像机时，照相机获取场景的一组图像。 集合的图像彼此重叠。 进行摄像机焦距的初始估计。 初始焦距可以是任何合理的焦距。 使用焦距的初始估计，将图像集合在存储器中以确定估计的初始合成长度。 焦距的下一个最佳估计来自初始估计的合成长度。 使用下一个最佳焦距估计重组该组图像。 迭代该过程直到焦距的连续估计之间的绝对差小于预定阈值以校准相机。 另外，合成图像集合的过程可以使用加权插值方案来减少由噪声和数字化效应引起的模糊效应和不正确的焦距。