公开(公告)号：US07463769B2

公开(公告)日：2008-12-09

申请号：US11156988

申请日：2005-06-20

申请人： Stephen S. Lin ,  Baining Guo ,  Heung-Yeung Shum ,  Jinwei Gu

发明人： Stephen S. Lin ,  Baining Guo ,  Heung-Yeung Shum ,  Jinwei Gu

IPC分类号： G06K9/00

CPC分类号： H04N1/603 ,  G06T7/80 ,  G06T7/90

摘要： Radiometric calibration of an image capture device (e.g., a digital camera) using a single image is described. The single image may be a color image or a grayscale image. The calibration identifies and analyzes edge pixels of the image that correspond to an edge between two colors or grayscale levels of a scene. Intensity distributions of intensities measured from the single image are then analyzed. An inverse response function for the image capture device is determined based on the intensity distributions. For a color image, the radiometric calibration involves calculating an inverse response function that maps measured blended colors of edge pixels and the associated measured component colors into linear distributions. For a grayscale image, the radiometric calibration involves deriving an inverse response function that maps non-uniform histograms of measured intensities into uniform distributions of calibrated intensities.

摘要翻译： 描述使用单个图像的图像捕获装置（例如，数码相机）的放射线校准。 单个图像可以是彩色图像或灰度图像。 校准识别和分析与场景的两种颜色或灰度级之间的边缘对应的图像的边缘像素。 然后分析从单个图像测量的强度的强度分布。 基于强度分布确定图像捕获装置的反应响应函数。 对于彩色图像，辐射校准包括计算反向响应函数，其将边缘像素的测量混合颜色和相关联的测量分量颜色映射为线性分布。 对于灰度图像，辐射校准涉及导出将测得的强度的不均匀直方图映射到校准强度的均匀分布的逆响应函数。

公开(公告)号：US07463296B2

公开(公告)日：2008-12-09

申请号：US10817008

申请日：2004-04-01

申请人： Jian Sun ,  Jiaya Jia ,  Heung-Yeung Shum

发明人： Jian Sun ,  Jiaya Jia ,  Heung-Yeung Shum

IPC分类号： H04N5/202 ,  H04N5/235

CPC分类号： H04N5/243 ,  G06T5/009 ,  G06T5/40 ,  G06T5/50 ,  G06T2207/10024 ,  G06T2207/10144 ,  H04N5/23248 ,  H04N5/2355

摘要： Techniques are disclosed to improve quality of images that may be blurred or underexposed (e.g., because of camera shake, taken in dim lighting conditions, or taken of high action scenes). The techniques may be implemented in a digital camera, digital video camera, or a digital camera capable of capturing video. In one described implementation, a digital camera includes an image sensor, a storage device, and a processing unit. The image sensor captures two images from a same scene which are stored on the storage device. The processing unit enhances the captured images with luminance correction.

摘要翻译： 公开了提高可能模糊或曝光不足的图像质量的技术（例如，由于相机抖动，在昏暗的照明条件下拍摄或拍摄的高动作场景）。 这些技术可以在能够捕获视频的数码相机，数字摄像机或数字照相机中实现。 在一个描述的实施方案中，数字照相机包括图像传感器，存储装置和处理单元。 图像传感器捕获存储在存储设备上的相同场景中的两个图像。 处理单元通过亮度校正来增强拍摄图像。

公开(公告)号：US07450754B2

公开(公告)日：2008-11-11

申请号：US10809167

申请日：2004-03-23

申请人： Stephen S. Lin ,  Heung-Yeung Shum ,  Jinwei Gu

发明人： Stephen S. Lin ,  Heung-Yeung Shum ,  Jinwei Gu

IPC分类号： G06K9/00

CPC分类号： H04N1/603 ,  G06T7/80 ,  G06T7/90

摘要： A radiometric calibration system finds an inverse response function of a camera from a single digital image of a scene in which the actual colors of the scene are not known a priori. The system analyzes pixels of the image that correspond to an “edge” between two colors of the scene. These “edge” pixels represent a blended color formed from these two “component” colors, as measured by the camera. The system determines an inverse response function at least in part by: (a) finding suitable edge pixels; and (b) determining a function that maps the measured blended colors of edge pixels and their measured component colors into linear distributions. Reference data that includes predetermined inverse response functions of known cameras can be used in determining an inverse response function via a Bayesian Estimation.

摘要翻译： 辐射校准系统从场景的实际颜色先前不知道的场景的单个数字图像中发现摄像机的反向响应功能。 该系统分析对应于场景两种颜色之间的“边缘”的图像像素。 这些“边缘”像素表示由相机测量的由这两个“分量”颜色形成的混合色。 系统至少部分地通过以下方式确定反应响应函数：（a）找到合适的边缘像素; 和（b）确定将所测量的边缘像素的混合颜色及其测量的分量颜色映射成线性分布的函数。 包括已知摄像机的预定反向响应功能的参考数据可以用于经由贝叶斯估计来确定反向响应函数。

公开(公告)号：US07433490B2

公开(公告)日：2008-10-07

申请号：US11435122

申请日：2006-05-16

申请人： Ying Huang ,  Stephen Ssu-te Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Ying Huang ,  Stephen Ssu-te Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06K9/00 ,  G10L15/00

CPC分类号： G06K9/00335 ,  G10L2021/105

摘要： A novel method for synchronizing the lips of a sketched face to an input voice. The lip synchronization system and method approach is to use training video as much as possible when the input voice is similar to the training voice sequences. Initially, face sequences are clustered from video segments, then by making use of sub-sequence Hidden Markov Models, a correlation between speech signals and face shape sequences is built. From this re-use of video, the discontinuity between two consecutive output faces is decreased and accurate and realistic synthesized animations are obtained. The lip synchronization system and method can synthesize faces from input audio in real-time without noticeable delay. Since acoustic feature data calculated from audio is directly used to drive the system without considering its phonemic representation, the method can adapt to any kind of voice, language or sound.

摘要翻译： 一种用于将草绘脸部的嘴唇同步到输入声音的新颖方法。 唇部同步系统和方法方法是当输入的声音类似于训练声音序列时尽可能多地使用训练视频。 最初，面部序列从视频片段聚类，然后通过利用子序列隐马尔可夫模型，构建了语音信号和面部形状序列之间的相关性。 从这种视频的再次使用，两个连续的输出面之间的不连续性被降低，并且获得准确而逼真的合成动画。 唇同步系统和方法可以实时地从输入音频合成面部，而没有明显的延迟。 由于从音频计算的声学特征数据直接用于驱动系统而不考虑其音位表示，该方法可以适应任何种类的语音，语言或声音。

公开(公告)号：US07408545B1

公开(公告)日：2008-08-05

申请号：US09560828

申请日：2000-04-28

申请人： Heung-Yeung Shum ,  Min-Sheng Wu ,  Hong-Hui Sun

发明人： Heung-Yeung Shum ,  Min-Sheng Wu ,  Hong-Hui Sun

IPC分类号： G06T17/00

CPC分类号： G06T15/205

摘要： With an image-based rendering (IBR) system based on linear interpolated concentric mosaics, an observer is able to wander through a real or synthesized scene and have new view images (optionally including stereo images) of the scene rendered as the observer moves. According to one embodiment, multiple viewing rays are extended in a viewing direction from the image to be rendered at a view point. For viewing rays that do not coincide with captured images, an image is generate based on the captured images. The image is generated by interpolating between at least two captured images based on a constant distance to objects in the scene. The view images that are displayed can be mono or stereo (e.g., simulating a pair of eyes of the observer).

摘要翻译： 利用基于线性内插同心马赛克的基于图像的渲染（IBR）系统，观察者能够通过真实或合成的场景漫游，并且随观察者移动而呈现的场景具有新的视图图像（可选地包括立体图像）。 根据一个实施例，多个观看光线在观看方向上从在观察点呈现的图像延伸。 为了观察与拍摄图像不一致的光线，基于捕获的图像生成图像。 基于与场景中的对象的恒定距离，在至少两个拍摄图像之间进行内插来生成图像。 显示的视图图像可以是单声道或立体声（例如，模拟观察者的一对眼睛）。

公开(公告)号：US20070263119A1

公开(公告)日：2007-11-15

申请号：US11434567

申请日：2006-05-15

申请人： Heung-Yeung Shum ,  Jian Sun ,  Sing Kang ,  Yin Li

发明人： Heung-Yeung Shum ,  Jian Sun ,  Sing Kang ,  Yin Li

IPC分类号： H04N5/222

CPC分类号： H04N5/275 ,  H04N5/2354

摘要： Foreground object matting uses flash/no-flash images pairs to obtain a flash-only image. A trimap is obtained from the flash-only image. A joint Bayesian algorithm uses the flash-only image, the trimap and one of the image of the scene taken without the flash or the image of the scene taken with the flash to generate a high quality matte that can be used to extract the foreground from the background.

摘要翻译： 前景对象消光使用闪光/非闪光图像对来获取闪光灯图像。 从闪光灯图像获得一个微调。 联合贝叶斯算法使用仅闪光图像，微调和没有闪光灯拍摄的场景中的一个图像或用闪光灯拍摄的场景的图像，以生成可用于从 的背景。

公开(公告)号：US20070247472A1

公开(公告)日：2007-10-25

申请号：US11277938

申请日：2006-03-29

申请人： Xin Tong ,  Yanyun Chen ,  Baining Guo ,  Heung-Yeung Shum ,  Jiaping Wang ,  John Snyder

发明人： Xin Tong ,  Yanyun Chen ,  Baining Guo ,  Heung-Yeung Shum ,  Jiaping Wang ,  John Snyder

IPC分类号： G09G5/00

CPC分类号： G01B11/24 ,  G01S17/89 ,  G06T15/04

摘要： A method and system for implementing capturing and rendering geometric details for mesostructure surfaces is described herein. A mesostructure distance function is defined as a function of a given reference point and a given viewing direction. A distance from a reference point to a mesostructure surface point along a viewing direction is measured using the mesostructure distance function. This distance is used to determine the visibility of mesostructure surface for rendering silhouettes. The lighting visibility of the mesostructure surface point may also be determined and used for determining whether the mesostructure surface point is in shadow. This determination may then be used for rendering shadow silhouettes.

摘要翻译： 本文描述了一种用于实现用于介观结构表面的捕获和渲染几何细节的方法和系统。 介观结构距离函数被定义为给定参考点和给定观察方向的函数。 使用介观结构距离函数测量从参考点到观察方向的介观结构表面点的距离。 该距离用于确定用于渲染轮廓的介观结构表面的可见性。 也可以确定介观结构表面点的照明可见度，并用于确定介观结构表面点是否处于阴影。 然后可以将该确定用于渲染阴影剪影。

公开(公告)号：US20070154110A1

公开(公告)日：2007-07-05

申请号：US11323358

申请日：2005-12-30

申请人： Fang Wen ,  Ying-Qing Xu ,  Heung-Yeung Shum ,  David Vronay

发明人： Fang Wen ,  Ying-Qing Xu ,  Heung-Yeung Shum ,  David Vronay

IPC分类号： G06K9/36 ,  G06K9/34

CPC分类号： G06T15/02

摘要： A non-photorealistic technique is described for transforming an original image into a sketch image. The technique includes: segmenting the original image into plural regions to produce a segmented image, wherein the regions are demarcated by respective boundaries; shrinking a boundary of at least one of the plural regions in the segmented image to produce a boundary-shrunk image; and modifying at least one color in the boundary-shrunk image to produce the sketch image.

摘要翻译： 描述了用于将原始图像变换成草图图像的非真实感技术。 该技术包括：将原始图像分割成多个区域以产生分割图像，其中通过相应边界划分区域; 缩小分割图像中的多个区域中的至少一个的边界以产生边界收缩图像; 并修改边界缩小图像中的至少一种颜色以产生草图图像。

公开(公告)号：US20070122001A1

公开(公告)日：2007-05-31

申请号：US11290135

申请日：2005-11-30

申请人： Qiang Wang ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

发明人： Qiang Wang ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

IPC分类号： G06K9/00

CPC分类号： G06K9/00208 ,  G06K9/00241

摘要： Systems and methods are described for real-time Bayesian 3D pose tracking. In one implementation, exemplary systems and methods formulate key-frame based differential pose tracking in a probabilistic graphical model. An exemplary system receives live captured video as input and tracks a video object's 3D pose in real-time based on the graphical model. An exemplary Bayesian inter-frame motion inference technique simultaneously performs online point matching and pose estimation. This provides robust pose tracking because the relative pose estimate for a current frame is simultaneously estimated from two independent sources, from a key-frame pool and from the video frame preceding the current frame. Then, an exemplary online Bayesian frame fusion technique infers the current pose from the two independent sources, providing stable and drift-free tracking, even during agile motion, occlusion, scale change, and drastic illumination change of the tracked object.

摘要翻译： 描述了实时贝叶斯3D姿态跟踪的系统和方法。 在一个实现中，示例性系统和方法在概率图形模型中制定基于关键帧的差分姿态跟踪。 示例性系统基于图形模型实时地接收实时捕获的视频作为输入并实时跟踪视频对象的3D姿态。 示例性的贝叶斯帧间运动推理技术同时执行在线点匹配和姿态估计。 这提供了鲁棒的姿势跟踪，因为当前帧的相对姿态估计是从两个独立的来源，从关键帧池和当前帧之前的视频帧同时估计的。 然后，示例性的在线贝叶斯帧融合技术从两个独立的来源推测出当前姿态，即使在敏捷运动，闭塞，比例变化和被跟踪对象的剧烈照明改变的情况下，也提供稳定和无漂移的跟踪。

公开(公告)号：US20070086622A1

公开(公告)日：2007-04-19

申请号：US11380635

申请日：2006-04-27

申请人： Jian Sun ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

发明人： Jian Sun ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

IPC分类号： G06K9/00 ,  G06K9/34

CPC分类号： G06K9/3241 ,  G06K9/32 ,  G06T7/277

摘要： The present video tracking technique outputs a Maximum A Posterior (MAP) solution for a target object based on two object templates obtained from a start and an end keyframe of a whole state sequence. The technique first minimizes the whole state space of the sequence by generating a sparse set of local two-dimensional modes in each frame of the sequence. The two-dimensional modes are converted into three-dimensional points within a three-dimensional volume. The three-dimensional points are clustered using a spectral clustering technique where each cluster corresponds to a possible trajectory segment of the target object. If there is occlusion in the sequence, occlusion segments are generated so that an optimal trajectory of the target object can be obtained.

摘要翻译： 本视频跟踪技术基于从整个状态序列的开始和结束关键帧获得的两个对象模板，为目标对象输出最大A后验（MAP）解决方案。 该技术首先通过在序列的每个帧中生成稀疏的局部二维模式集来最小化序列的整个状态空间。 二维模式在三维体积内被转换成三维点。 使用光谱聚类技术对三维点进行聚类，其中每个聚类对应于目标对象的可能的轨迹段。 如果序列中存在闭塞，则生成闭塞段，从而可以获得目标对象的最佳轨迹。