公开(公告)号：US07343039B2

公开(公告)日：2008-03-11

申请号：US10660819

申请日：2003-09-09

Applicant: Xinguo Liu ,  Sing Bing Kang ,  Heung-Yeung Shum

Inventor： Xinguo Liu ,  Sing Bing Kang ,  Heung-Yeung Shum

IPC: G06K9/00

CPC classification number: G06K9/50 ,  G06K9/00201 ,  G06K9/4647

Abstract: A system and process for determining the similarity in the shape of objects is presented that generates a novel shape representation called a directional histogram model. This shape representative captures the shape variations of an object with viewing direction, using thickness histograms. The resulting directional histogram model is substantially invariant to scaling and translation. A matrix descriptor can also be derived by applying the spherical harmonic transform to the directional histogram model. The resulting matrix descriptor is substantially invariant to not only scaling and translation, but rotation as well. The matrix descriptor is also robust with respect to local modification or noise, and able to readily distinguish objects with different global shapes. The typical applications of the directional histogram model and matrix descriptor include recognizing 3D solid shapes, measuring the similarity between different objects and shape similarity based object retrieval.

Abstract translation: 提出了一种用于确定物体形状相似度的系统和过程，其产生称为方向直方图模型的新颖形状表示。 该形状代表使用厚度直方图捕获具有观察方向的对象的形状变化。 所得到的方向直方图模型对缩放和平移基本上是不变的。 还可以通过将球谐函数变换应用于方向直方图模型来导出矩阵描述符。 所得到的矩阵描述符不仅不仅缩放和平移，而且旋转也是不变的。 矩阵描述符在本地修改或噪声方面也是鲁棒的，并且能够容易地区分具有不同全局形状的对象。 方向直方图模型和矩阵描述符的典型应用包括识别3D实体形状，测量不同对象之间的相似度和基于形状相似度的对象检索。

公开(公告)号：US07129949B2

公开(公告)日：2006-10-31

申请号：US10980760

申请日：2004-11-02

Applicant: Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

Inventor： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

IPC: G06T15/00 ,  G06T15/50

CPC classification number: G06T13/40 ,  G06T15/04 ,  G06T17/10 ,  G06T17/20 ,  G06T17/205 ,  G06T2200/08 ,  Y10S345/956

Abstract: The illustrated and described embodiments describe techniques for capturing data that describes 3-dimensional (3-D) aspects of a face, transforming facial motion from one individual to another in a realistic manner, and modeling skin reflectance.

Abstract translation: 所示出和描述的实施例描述了用于捕获描述面部的三维（3-D）方面的数据的技术，以现实的方式将面部运动从一个人转换到另一个个体，以及对皮肤反射率进行建模。

公开(公告)号：US07010174B2

公开(公告)日：2006-03-07

申请号：US10965935

申请日：2004-10-15

Applicant: Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder ,  Richard Szeliski

Inventor： Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder ,  Richard Szeliski

IPC: G06K9/40

CPC classification number: H04N5/2355 ,  G06T5/50 ,  H04N5/235 ,  H04N5/2352 ,  H04N5/77 ,  H04N5/781 ,  H04N5/85

Abstract: A system and process for generating High Dynamic Range (HDR) video is presented which involves first capturing a video image sequence while varying the exposure so as to alternate between frames having a shorter and longer exposure. The exposure for each frame is set prior to it being captured as a function of the pixel brightness distribution in preceding frames. Next, for each frame of the video, the corresponding pixels between the frame under consideration and both preceding and subsequent frames are identified. For each corresponding pixel set, at least one pixel is identified as representing a trustworthy pixel. The pixel color information associated with the trustworthy pixels is then employed to compute a radiance value for each pixel set to form a radiance map. A tone mapping procedure can then be performed to convert the radiance map into an 8-bit representation of the HDR frame.

Abstract translation: 提出了用于产生高动态范围（HDR）视频的系统和过程，其涉及首先在改变曝光的同时捕获视频图像序列，以便在具有较短和较长曝光的帧之间交替。 每个帧的曝光在其被捕获之前被设置为在先前帧中的像素亮度分布的函数。 接下来，对于视频的每个帧，识别所考虑的帧与前后帧之间的对应像素。 对于每个对应的像素集合，至少一个像素被识别为表示可靠的像素。 然后使用与可信赖像素相关联的像素颜色信息来计算每个像素组的辐射值以形成辐射图。 然后可以执行色调映射过程以将辐射图转换成HDR帧的8位表示。

公开(公告)号：US07006709B2

公开(公告)日：2006-02-28

申请号：US10172859

申请日：2002-06-15

Applicant: Sing Bing Kang ,  Richard S. Szeliski ,  Matthew T. Uyttendaele

Inventor： Sing Bing Kang ,  Richard S. Szeliski ,  Matthew T. Uyttendaele

IPC: G06K9/32

CPC classification number: G06T5/20 ,  G06K9/32 ,  G06K2009/2045 ,  G06T3/00 ,  G06T5/003 ,  G06T7/30 ,  G06T7/55 ,  G06T15/503

Abstract: A system and method for deghosting mosaics provides a novel multiperspective plane sweep approach for generating an image mosaic from a sequence of still images, video images, scanned photographic images, computer generated images, etc. This multiperspective plane sweep approach uses virtual camera positions to compute depth maps for columns of overlapping pixels in adjacent images. Object distortions and ghosting caused by image parallax when generating the image mosaics are then minimized by blending pixel colors, or grey values, for each computed depth to create a common composite area for each of the overlapping images. Further, the multiperspective plane sweep approach described herein is both computationally efficient, and applicable to both the case of limited overlap between the images used for creating the image mosaics, and to the case of extensive or increased image overlap.

公开(公告)号：US06950104B1

公开(公告)日：2005-09-27

申请号：US09651880

申请日：2000-08-30

Applicant: Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

Inventor： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

IPC: G06T1/00 ,  G06T7/20 ,  G06T13/40 ,  G06T15/04 ,  G06T17/10 ,  G06T17/20 ,  G06T15/70

CPC classification number: G06T13/40 ,  G06T15/04 ,  G06T17/10 ,  G06T17/20 ,  G06T17/205 ,  G06T2200/08 ,  Y10S345/956

Abstract: Methods and systems for animating facial features and transforming facial expressions are described. In one embodiment, a code book contains data that defines a set of facial expressions of a first person. A training set of facial expressions from a second person and corresponding expressions from the code book are used to derive a transformation function that is then applied to all of the expressions of the code book. In this manner, expressions from the first person can be realistically transformed into expressions of a second person and vice versa. Particularly advantageous aspects of the described embodiments provide a single common generic face model that is used as the basis for a fitting operation for many different faces. Use of the single common generic face model and certain user-defined constraints provide a mechanism by which correspondences between the different faces can be established. These correspondences provide a basis for facial animation operations, among which are included expression transformation.

Abstract translation: 描述了用于动画面部特征和变换面部表情的方法和系统。 在一个实施例中，代码簿包含定义第一人的一组面部表情的数据。 使用来自第二人的面部表情的训练集和来自代码簿的对应表达式来导出转换函数，然后将其应用于代码簿的所有表达式。 以这种方式，来自第一人的表达可以被实际地转换成第二人的表达，反之亦然。 所描述的实施例的特别有利的方面提供了单个公共通用面部模型，其被用作许多不同面部的装配操作的基础。 使用单一公共通用面部模型和某些用户定义的约束提供了可以建立不同面部之间的对应关系的机制。 这些通信为面部动画操作提供了基础，其中包括表达式转换。

公开(公告)号：US06870563B1

公开(公告)日：2005-03-22

申请号：US09591781

申请日：2000-06-12

Applicant: Sing Bing Kang

Inventor： Sing Bing Kang

IPC: G06T5/00 ,  G06T7/00 ,  H04N5/225 ,  H04N5/228 ,  H04N5/262

CPC classification number: H04N5/2628 ,  G06T5/006 ,  G06T7/80

Abstract: A method and a system for self-calibrating a wide field-of-view camera (such as a catadioptric camera) using a sequence of omni-directional images of a scene obtained from the camera. The present invention uses the consistency of pairwise features tracked across at least a portion of the image collection and uses these tracked features to determine unknown calibration parameters based on the characteristics of catadioptric imaging. More specifically, the self-calibration method of the present invention generates a sequence of omni-directional images representing a scene and tracks features across the image sequence. An objective function is defined in terms of the tracked features and an error metric (an image-based error metric in a preferred embodiment). The catadioptric imaging characteristics are defined by calibration parameters, and determination of optimal calibration parameters is accomplished by minimizing the objective function using an optimizing technique. Moreover, the present invention also includes a technique for reformulating a projection equation such that the projection equation is equivalent to that of a rectilinear perspective camera. This technique allows analyses (such as structure from motion) to be applied (subsequent to calibration of the catadioptric camera) in the same direct manner as for rectilinear image sequences.

Abstract translation: 一种用于使用从相机获得的场景的全方位图像序列来自动校准宽视野相机（例如反折射相机）的方法和系统。 本发明使用在图像集合的至少一部分上跟踪的成对特征的一致性，并且使用这些跟踪的特征来基于反射折射成像的特征来确定未知的校准参数。 更具体地，本发明的自校准方法生成表示场景的全方位图像序列，并且跨越图像序列跟踪特征。 根据跟踪特征和误差度量（优选实施例中的基于图像的误差度量）来定义目标函数。 反射折射成像特征由校准参数定义，并且通过使用优化技术来最小化目标函数来实现最佳校准参数的确定。 此外，本发明还包括一种用于重新配置投影方程式的技术，使得投影方程相当于直线透视照相机的投影方程。 这种技术允许以与直线图像序列相同的直接方式应用分析（例如运动结构）（在反折射相机的校准之后）。

公开(公告)号：US06417850B1

公开(公告)日：2002-07-09

申请号：US09238250

申请日：1999-01-27

Applicant: Sing Bing Kang

Inventor： Sing Bing Kang

IPC: G06T1540

CPC classification number: G06T15/205

Abstract: A 3-D effect is added to a single image by adding depth to the single image. Depth can be added to the single image by selecting an arbitrary region or a number of pixels. A user interface simultaneously displays the single image and novel views of the single original image taken from virtual camera positions rotated relative to the original field of view. Depths given to the original image allow pixels to be reprojected onto the novel views to allow the user to observe the depth changes as they are being added. Functions are provided to edit gaps or voids generated in the process of adding depth to the single image. The gaps occur because of depth discontinuities between regions to which depth has been added and the voids are due to the uncovering of previously occluded surfaces in the original image.

Abstract translation: 通过向单个图像添加深度，将3-D效果添加到单个图像。 可以通过选择任意区域或多个像素将深度添加到单个图像。 用户界面同时显示从相对于原始视野旋转的虚拟摄像机位置拍摄的单个原始图像的单个图像和新颖视图。 给予原始图像的深度允许将像素重新投影到新颖视图上，以允许用户在添加时观察深度变化。 提供了功能来编辑在单个图像添加深度的过程中生成的空白或空白。 由于在添加了深度的区域之间的深度不连续性，并且空隙是由于原始图像中先前遮挡的表面的揭开而发生的。

公开(公告)号：US06295377B1

公开(公告)日：2001-09-25

申请号：US09114684

申请日：1998-07-13

Applicant: Frederic Dufaux ,  Sing Bing Kang

Inventor： Frederic Dufaux ,  Sing Bing Kang

IPC: G06K936

CPC classification number: G06T9/004 ,  G06T7/238 ,  G06T2207/10016

Abstract: In an coder for producing a bitstream representative of a sequence of video images, a previous image is registered with a current image using spline-based registration to produce estimated motion vectors. The estimated motion vectors are used to match blocks of the previous image and the current image to produce translation vectors. The translation vectors compensate for motion while encoding the sequence as a bitstream.

Abstract translation: 在用于产生表示视频图像序列的比特流的编码器中，使用基于样条的注册来对当前图像登记先前的图像，以产生估计的运动矢量。 估计的运动矢量用于匹配先前图像和当前图像的块以产生平移矢量。 翻译矢量补偿运动，同时将序列编码为比特流。

公开(公告)号：US06266068B1

公开(公告)日：2001-07-24

申请号：US09039022

申请日：1998-03-13

Applicant: Sing Bing Kang ,  James M. Rehg

Inventor： Sing Bing Kang ,  James M. Rehg

IPC: G06T1160

CPC classification number: G06T11/60

Abstract: A computerized method and related computer system synthesize video from a plurality of sources of image data. The sources include a variety of image data types such a collection of image stills, a sequence of video frames, and 3-D models of objects. Each source provides image data associated with an object. One source provides image data associated with a first object, and a second source provides image data associated with a second object. The image data of the first and second objects are combined to generate composite images of the first and second objects. From the composite images, an output image of the first and second objects as viewed from an arbitrary viewpoint is generated. Gaps of pixels with unspecified pixel values may appear in the output image. Accordingly, a pixel value for each of these “missing pixels” is obtained by using an epipolar search process to determine which one of the sources of image data should provide the pixel value for that missing pixel.

Abstract translation: 计算机化方法和相关计算机系统从多个图像数据源合成视频。 这些源包括各种图像数据类型，诸如图像静止图像集合，视频帧序列和对象的3D模型。 每个源提供与对象相关联的图像数据。 一个源提供与第一对象相关联的图像数据，第二源提供与第二对象相关联的图像数据。 组合第一和第二对象的图像数据以生成第一和第二对象的合成图像。 从合成图像生成从任意视点观察第一和第二对象的输出图像。 输出图像中可能会出现具有未指定像素值的像素间隙。 因此，通过使用对极搜索处理来确定图像数据的哪一个源应该为缺失像素提供像素值，从而获得每个这些“丢失像素”的像素值。

公开(公告)号：US6061468A

公开(公告)日：2000-05-09

申请号：US901391

申请日：1997-07-28

Applicant: Sing Bing Kang

Inventor： Sing Bing Kang

IPC: G06T1/00 ,  G06T7/20 ,  G06T17/40 ,  G06K9/00

CPC classification number: G06T7/0071 ,  G06T2207/10016

Abstract: In a computerized method, the three-dimensional structure of an object is recovered from a closed-loop sequence of two-dimensional images taken by a camera undergoing some arbitrary motion. In one type of motion, the camera is held fixed, while the object completes a full 360.degree. rotation about an arbitrary axis. Alternatively, the camera can make a complete rotation about the object. In the sequence of images, feature tracking points are selected using pair-wise image registration. Ellipses are fitted to the feature tracking points to estimate the tilt of the axis of rotation. A set of variables are set to fixed values while minimizing an image-based objective function to extract a set of first structure and motion parameters. Then the set of variables freed while minimizing of the objective function continues to extract a second set of structure and motion parameters that are substantially the same as the first set of structure and motion parameters.

Abstract translation: 在计算机化方法中，从经历某种任意运动的相机拍摄的二维图像的闭环序列中恢复对象的三维结构。 在一种类型的运动中，摄像机保持固定，同时物体围绕任意轴完成360度旋转。 或者，相机可以围绕对象进行完全旋转。 在图像序列中，使用成对图像配准来选择特征跟踪点。 椭圆适配到特征跟踪点以估计旋转轴的倾斜度。 将一组变量设置为固定值，同时使基于图像的目标函数最小化以提取一组第一结构和运动参数。 然后在最小化目标函数的同时释放的变量集继续提取与第一组结构和运动参数基本相同的第二组结构和运动参数。