公开(公告)号：US09275680B2

公开(公告)日：2016-03-01

申请号：US12485179

申请日：2009-06-16

申请人： Charles Lawrence Zitnick, III ,  Bryan K. Ressler ,  Sing Bing Kang ,  Michael F. Cohen ,  Jagannatha Koppal

发明人： Charles Lawrence Zitnick, III ,  Bryan K. Ressler ,  Sing Bing Kang ,  Michael F. Cohen ,  Jagannatha Koppal

IPC分类号： G06F3/048 ,  G11B27/034 ,  G11B27/34 ,  H04N21/485 ,  H04N21/6377 ,  H04N21/658

CPC分类号： G11B27/034 ,  G11B27/34 ,  H04N21/4854 ,  H04N21/6377 ,  H04N21/658

摘要： Described is a user interface that displays a representation of a stereo scene, and includes interactive mechanisms for changing parameter values that determine the perceived appearance of that scene. The scene is modeled as if viewed from above, including a representation of a viewer's eyes, a representation of a viewing screen, and an indication simulating what each of the viewer eyes perceives on the viewing screen. Variable parameters may include a vergence parameter, a dolly parameter, a field-of-view parameter, an interocular parameter and a proscenium arch parameter.

摘要翻译： 描述了显示立体场景的表示的用户界面，并且包括用于改变确定该场景的感知外观的参数值的交互机制。 该场景被建模为仿佛从上方观看，包括观看者的眼睛的表示，观看屏幕的表示，以及模拟观看者眼睛在观看屏幕上感知的每一个的指示。 可变参数可以包括聚集参数，小车参数，视野参数，眼镜参数和前景拱参数。

公开(公告)号：US20130095920A1

公开(公告)日：2013-04-18

申请号：US13273213

申请日：2011-10-13

申请人： Kestutis Patiejunas ,  Kanchan Mitra ,  Patrick Sweeney ,  Yaron Eshet ,  Adam G. Kirk ,  Sing Bing Kang ,  Charles Lawrence Zitnick, III ,  David Eraker ,  David Harnett ,  Amit Mital ,  Simon Winder

发明人： Kestutis Patiejunas ,  Kanchan Mitra ,  Patrick Sweeney ,  Yaron Eshet ,  Adam G. Kirk ,  Sing Bing Kang ,  Charles Lawrence Zitnick, III ,  David Eraker ,  David Harnett ,  Amit Mital ,  Simon Winder

IPC分类号： A63F13/00 ,  H04N13/02

CPC分类号： G06T15/00 ,  G06T7/521 ,  G06T7/593 ,  G06T15/04 ,  G06T17/20 ,  G06T2207/10021 ,  G06T2207/10024 ,  G06T2207/10048 ,  G06T2207/20228 ,  H04N13/111 ,  H04N13/271 ,  H04N2013/0081

摘要： Methods and systems for generating free viewpoint video using an active infrared (IR) stereo module are provided. The method includes computing a depth map for a scene using an active IR stereo module. The depth map may be computed by projecting an IR dot pattern onto the scene, capturing stereo images from each of two or more synchronized IR cameras, detecting dots within the stereo images, computing feature descriptors corresponding to the dots in the stereo images, computing a disparity map between the stereo images, and generating the depth map using the disparity map. The method also includes generating a point cloud for the scene using the depth map, generating a mesh of the point cloud, and generating a projective texture map for the scene from the mesh of the point cloud. The method further includes generating the video for the scene using the projective texture map.

摘要翻译： 提供了使用主动红外（IR）立体声模块产生免费视点视频的方法和系统。 该方法包括使用主动IR立体声模块来计算场景的深度图。 可以通过将IR点图案投影到场景上来计算深度图，从两个或更多个同步红外相机中的每一个拍摄立体图像，检测立体图像内的点，计算与立体图像中的点相对应的特征描述符， 立体图像之间的视差图，并使用视差图生成深度图。 该方法还包括使用深度图生成场景的点云，生成点云的网格，并从点云的网格生成场景的投影纹理贴图。 该方法还包括使用投影纹理图生成场景的视频。

公开(公告)号：US07884823B2

公开(公告)日：2011-02-08

申请号：US11761604

申请日：2007-06-12

申请人： Joe Bertolami ,  Robert M. Craig ,  Dax Hawkins ,  Sing Bing Kang ,  Jonathan E. Lange

发明人： Joe Bertolami ,  Robert M. Craig ,  Dax Hawkins ,  Sing Bing Kang ,  Jonathan E. Lange

IPC分类号： G06T15/20 ,  G06T15/00

CPC分类号： A63F13/525 ,  A63F13/10 ,  A63F2300/66 ,  G06T15/20 ,  H04N13/275

摘要： Game data is rendered in three dimensions in the GPU of a game console. A left camera view and a right camera view are generated from a single camera view. The left and right camera positions are derived as an offset from a default camera. The focal distance of the left and right cameras is infinity. A game developer does not have to encode dual images into a specific hardware format. When a viewer sees the two slightly offset images, the user's brain combines the two offset images into a single 3D image to give the illusion that objects either pop out from or recede into the display screen. In another embodiment, individual, private video is rendered, on a single display screen, for different viewers. Rather than rendering two similar offset images, two completely different images are rendered allowing each player to view only one of the images.

摘要翻译： 游戏数据在游戏机的GPU中呈现三维。 从单个摄像机视图生成左侧摄像机视图和右侧摄像机视图。 左和右摄像机位置被派生为与默认摄像机的偏移量。 左右相机的焦距为无穷远。 游戏开发者不必将双重图像编码为特定的硬件格式。 当观众看到两个轻微偏移的图像时，用户的大脑将两个偏移图像组合成单个3D图像，以给出对象从显示屏幕中弹出或退回到显示屏幕的错觉。 在另一个实施例中，单独的专用视频在单个显示屏幕上被呈现给不同的观看者。 而不是渲染两个相似的偏移图像，渲染两个完全不同的图像，允许每个播放器仅查看其中一个图像。

公开(公告)号：US07808532B2

公开(公告)日：2010-10-05

申请号：US11807448

申请日：2007-05-29

申请人： Jian Sun ,  Jian Sun ,  Sing Bing Kang ,  Xiaoou Tang ,  Heung-Yeung Shum

发明人： Jian Sun ,  Jian Sun ,  Sing Bing Kang ,  Xiaoou Tang ,  Heung-Yeung Shum

IPC分类号： H04N9/73

CPC分类号： H04N9/76 ,  G06T7/11 ,  G06T7/143 ,  G06T7/194 ,  G06T2207/10144 ,  H04N5/23232

摘要： A flash-based strategy is used to separate foreground information from background information within image information. In this strategy, a first image is taken without the use of flash. A second image is taken of the same subject matter with the use of flash. The foreground information in the flash image is illuminated by the flash to a much greater extent than the background information. Based on this property, the strategy applies processing to extract the foreground information from the background information. The strategy supplements the flash information by also taking into consideration motion information and color information.

摘要翻译： 基于闪存的策略用于将前景信息与图像信息中的背景信息分离。 在这个策略中，第一个图像是不使用闪光灯的。 使用闪光灯拍摄相同主题的第二张照片。 闪光灯中的前景信息被闪光灯照亮到比背景信息更大的程度。 基于此属性，该策略应用处理从背景信息中提取前景信息。 该策略通过考虑运动信息和颜色信息来补充闪光信息。

公开(公告)号：US20100111417A1

公开(公告)日：2010-05-06

申请号：US12263618

申请日：2008-11-03

申请人： Benjamin Ward ,  Sing Bing Kang ,  Eric Bennett

发明人： Benjamin Ward ,  Sing Bing Kang ,  Eric Bennett

IPC分类号： G06K9/34

CPC分类号： G06T7/579 ,  G06T2207/10012 ,  G06T2207/10016 ,  G06T2207/20104 ,  H04N13/261

摘要： Two-dimensional (2D) video is converted into multi-view video. The 2D video is segmented to generate a temporally consistent segmented 2D video which is made up of a sequence of segmented frames. The multi-view video is generated by employing user-guided operations to generate depth assignments for the segments associated with user-assigned regions of the segmented frames, where a user-assigned region is formed from a group of contiguous segments selected by the user.

摘要翻译： 二维（2D）视频转换为多视角视频。 2D视频被分割以产生由分段帧序列组成的时间上一致的分割的2D视频。 多视点视频是通过采用用户指导的操作来生成与分段帧的用户分配区域相关联的片段的深度分配，其中由用户选择的一组连续片段形成用户分配的区域。

公开(公告)号：US07561620B2

公开(公告)日：2009-07-14

申请号：US10910077

申请日：2004-08-03

申请人： Simon Winder ,  Matthew Uyttendaele ,  Charles Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang

发明人： Simon Winder ,  Matthew Uyttendaele ,  Charles Zitnick, III ,  Richard Szeliski ,  Sing Bing Kang

IPC分类号： H04B1/66 ,  H04N7/12 ,  H04N11/02 ,  H04N11/04 ,  H04N5/14 ,  H04N9/64

CPC分类号： H04N21/4347 ,  H04N13/111 ,  H04N19/109 ,  H04N19/39 ,  H04N19/593 ,  H04N19/597 ,  H04N19/61 ,  H04N19/70 ,  H04N19/96 ,  H04N21/2365

摘要： A system and process for compressing and decompressing multiple video streams depicting substantially the same dynamic scene from different viewpoints. Each frame in each contemporaneous set of video frames of the multiple streams is represented by at least a two layers—a main layer and a boundary layer. Compression of the main layers involves first designating one or more of these layers in each set of contemporaneous frames as keyframes. For each set of contemporaneous frames in time sequence order, the main layer of each keyframe is compressed using an inter-frame compression technique. In addition, the main layer of each non-keyframe within the frame set under consideration is compressed using a spatial prediction compression technique. Finally, the boundary layers of each frame in the current frame set are each compressed using an intra-frame compression technique. Decompression is generally the reverse of the compression process.

摘要翻译： 一种用于压缩和解压缩从不同观点描绘基本相同的动态场景的多个视频流的系统和过程。 多个流的每个同期视频帧集合中的每个帧由至少两层（主层和边界层）表示。 主要层的压缩包括首先将每组同期帧中的这些层中的一个或多个指定为关键帧。 对于按时间顺序排列的每组同期帧，使用帧间压缩技术对每个关键帧的主层进行压缩。 另外，使用空间预测压缩技术对所考虑的帧集合内的每个非关键帧的主层进行压缩。 最后，使用帧内压缩技术对当前帧集合中每帧的边界层进行压缩。 压缩通常与压缩过程相反。

公开(公告)号：US07343039B2

公开(公告)日：2008-03-11

申请号：US10660819

申请日：2003-09-09

申请人： Xinguo Liu ,  Sing Bing Kang ,  Heung-Yeung Shum

发明人： Xinguo Liu ,  Sing Bing Kang ,  Heung-Yeung Shum

IPC分类号： G06K9/00

CPC分类号： G06K9/50 ,  G06K9/00201 ,  G06K9/4647

摘要： A system and process for determining the similarity in the shape of objects is presented that generates a novel shape representation called a directional histogram model. This shape representative captures the shape variations of an object with viewing direction, using thickness histograms. The resulting directional histogram model is substantially invariant to scaling and translation. A matrix descriptor can also be derived by applying the spherical harmonic transform to the directional histogram model. The resulting matrix descriptor is substantially invariant to not only scaling and translation, but rotation as well. The matrix descriptor is also robust with respect to local modification or noise, and able to readily distinguish objects with different global shapes. The typical applications of the directional histogram model and matrix descriptor include recognizing 3D solid shapes, measuring the similarity between different objects and shape similarity based object retrieval.

摘要翻译： 提出了一种用于确定物体形状相似度的系统和过程，其产生称为方向直方图模型的新颖形状表示。 该形状代表使用厚度直方图捕获具有观察方向的对象的形状变化。 所得到的方向直方图模型对缩放和平移基本上是不变的。 还可以通过将球谐函数变换应用于方向直方图模型来导出矩阵描述符。 所得到的矩阵描述符不仅不仅缩放和平移，而且旋转也是不变的。 矩阵描述符在本地修改或噪声方面也是鲁棒的，并且能够容易地区分具有不同全局形状的对象。 方向直方图模型和矩阵描述符的典型应用包括识别3D实体形状，测量不同对象之间的相似度和基于形状相似度的对象检索。

公开(公告)号：US07129949B2

公开(公告)日：2006-10-31

申请号：US10980760

申请日：2004-11-02

申请人： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

发明人： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

IPC分类号： G06T15/00 ,  G06T15/50

CPC分类号： G06T13/40 ,  G06T15/04 ,  G06T17/10 ,  G06T17/20 ,  G06T17/205 ,  G06T2200/08 ,  Y10S345/956

摘要： The illustrated and described embodiments describe techniques for capturing data that describes 3-dimensional (3-D) aspects of a face, transforming facial motion from one individual to another in a realistic manner, and modeling skin reflectance.

摘要翻译： 所示出和描述的实施例描述了用于捕获描述面部的三维（3-D）方面的数据的技术，以现实的方式将面部运动从一个人转换到另一个个体，以及对皮肤反射率进行建模。

公开(公告)号：US07010174B2

公开(公告)日：2006-03-07

申请号：US10965935

申请日：2004-10-15

申请人： Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder ,  Richard Szeliski

发明人： Sing Bing Kang ,  Matthew T. Uyttendaele ,  Simon Winder ,  Richard Szeliski

IPC分类号： G06K9/40

CPC分类号： H04N5/2355 ,  G06T5/50 ,  H04N5/235 ,  H04N5/2352 ,  H04N5/77 ,  H04N5/781 ,  H04N5/85

摘要： A system and process for generating High Dynamic Range (HDR) video is presented which involves first capturing a video image sequence while varying the exposure so as to alternate between frames having a shorter and longer exposure. The exposure for each frame is set prior to it being captured as a function of the pixel brightness distribution in preceding frames. Next, for each frame of the video, the corresponding pixels between the frame under consideration and both preceding and subsequent frames are identified. For each corresponding pixel set, at least one pixel is identified as representing a trustworthy pixel. The pixel color information associated with the trustworthy pixels is then employed to compute a radiance value for each pixel set to form a radiance map. A tone mapping procedure can then be performed to convert the radiance map into an 8-bit representation of the HDR frame.

摘要翻译： 提出了用于产生高动态范围（HDR）视频的系统和过程，其涉及首先在改变曝光的同时捕获视频图像序列，以便在具有较短和较长曝光的帧之间交替。 每个帧的曝光在其被捕获之前被设置为在先前帧中的像素亮度分布的函数。 接下来，对于视频的每个帧，识别所考虑的帧与前后帧之间的对应像素。 对于每个对应的像素集合，至少一个像素被识别为表示可靠的像素。 然后使用与可信赖像素相关联的像素颜色信息来计算每个像素组的辐射值以形成辐射图。 然后可以执行色调映射过程以将辐射图转换成HDR帧的8位表示。

公开(公告)号：US06950104B1

公开(公告)日：2005-09-27

申请号：US09651880

申请日：2000-08-30

申请人： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

发明人： Stephen Marschner ,  Brian K. Guenter ,  Sashi Raghupathy ,  Kirk Olynyk ,  Sing Bing Kang

IPC分类号： G06T1/00 ,  G06T7/20 ,  G06T13/40 ,  G06T15/04 ,  G06T17/10 ,  G06T17/20 ,  G06T15/70

CPC分类号： G06T13/40 ,  G06T15/04 ,  G06T17/10 ,  G06T17/20 ,  G06T17/205 ,  G06T2200/08 ,  Y10S345/956

摘要： Methods and systems for animating facial features and transforming facial expressions are described. In one embodiment, a code book contains data that defines a set of facial expressions of a first person. A training set of facial expressions from a second person and corresponding expressions from the code book are used to derive a transformation function that is then applied to all of the expressions of the code book. In this manner, expressions from the first person can be realistically transformed into expressions of a second person and vice versa. Particularly advantageous aspects of the described embodiments provide a single common generic face model that is used as the basis for a fitting operation for many different faces. Use of the single common generic face model and certain user-defined constraints provide a mechanism by which correspondences between the different faces can be established. These correspondences provide a basis for facial animation operations, among which are included expression transformation.

摘要翻译： 描述了用于动画面部特征和变换面部表情的方法和系统。 在一个实施例中，代码簿包含定义第一人的一组面部表情的数据。 使用来自第二人的面部表情的训练集和来自代码簿的对应表达式来导出转换函数，然后将其应用于代码簿的所有表达式。 以这种方式，来自第一人的表达可以被实际地转换成第二人的表达，反之亦然。 所描述的实施例的特别有利的方面提供了单个公共通用面部模型，其被用作许多不同面部的装配操作的基础。 使用单一公共通用面部模型和某些用户定义的约束提供了可以建立不同面部之间的对应关系的机制。 这些通信为面部动画操作提供了基础，其中包括表达式转换。