公开(公告)号：US20070263119A1

公开(公告)日：2007-11-15

申请号：US11434567

申请日：2006-05-15

申请人： Heung-Yeung Shum ,  Jian Sun ,  Sing Kang ,  Yin Li

发明人： Heung-Yeung Shum ,  Jian Sun ,  Sing Kang ,  Yin Li

IPC分类号： H04N5/222

CPC分类号： H04N5/275 ,  H04N5/2354

摘要： Foreground object matting uses flash/no-flash images pairs to obtain a flash-only image. A trimap is obtained from the flash-only image. A joint Bayesian algorithm uses the flash-only image, the trimap and one of the image of the scene taken without the flash or the image of the scene taken with the flash to generate a high quality matte that can be used to extract the foreground from the background.

摘要翻译： 前景对象消光使用闪光/非闪光图像对来获取闪光灯图像。 从闪光灯图像获得一个微调。 联合贝叶斯算法使用仅闪光图像，微调和没有闪光灯拍摄的场景中的一个图像或用闪光灯拍摄的场景的图像，以生成可用于从 的背景。

公开(公告)号：US20070086622A1

公开(公告)日：2007-04-19

申请号：US11380635

申请日：2006-04-27

申请人： Jian Sun ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

发明人： Jian Sun ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

IPC分类号： G06K9/00 ,  G06K9/34

CPC分类号： G06K9/3241 ,  G06K9/32 ,  G06T7/277

摘要： The present video tracking technique outputs a Maximum A Posterior (MAP) solution for a target object based on two object templates obtained from a start and an end keyframe of a whole state sequence. The technique first minimizes the whole state space of the sequence by generating a sparse set of local two-dimensional modes in each frame of the sequence. The two-dimensional modes are converted into three-dimensional points within a three-dimensional volume. The three-dimensional points are clustered using a spectral clustering technique where each cluster corresponds to a possible trajectory segment of the target object. If there is occlusion in the sequence, occlusion segments are generated so that an optimal trajectory of the target object can be obtained.

摘要翻译： 本视频跟踪技术基于从整个状态序列的开始和结束关键帧获得的两个对象模板，为目标对象输出最大A后验（MAP）解决方案。 该技术首先通过在序列的每个帧中生成稀疏的局部二维模式集来最小化序列的整个状态空间。 二维模式在三维体积内被转换成三维点。 使用光谱聚类技术对三维点进行聚类，其中每个聚类对应于目标对象的可能的轨迹段。 如果序列中存在闭塞，则生成闭塞段，从而可以获得目标对象的最佳轨迹。

公开(公告)号：US20070013813A1

公开(公告)日：2007-01-18

申请号：US11183226

申请日：2005-07-15

申请人： Jian Sun ,  Jiaya Jia ,  Heung-Yeung Shum

发明人： Jian Sun ,  Jiaya Jia ,  Heung-Yeung Shum

IPC分类号： H04N9/74

CPC分类号： G06T7/12 ,  G06T7/194 ,  G06T2207/20092

摘要： An exemplary method includes receiving a trimap for an image that specifies a background region, a foreground region and an unknown region for the image wherein a boundary exists between the foreground region and the unknown region and wherein another boundary exists between the unknown region and the background region, solving a set of Poisson equations having boundary conditions for the two boundaries to provide a matte that distinguishes a foreground region from a background region in the unknown region, and refining the matte by solving a set of Poisson equations for a local unknown region. Various other exemplary technologies are also presented.

摘要翻译： 一种示例性方法包括：接收用于图像的微调，该图像指定用于图像的背景区域，前景区域和未知区域，其中在前景区域和未知区域之间存在边界，并且其中在未知区域和背景之间存在另一边界 区域，求解一组具有两个边界的边界条件的泊松方程，以提供区分未知区域中的前景区域和背景区域的无光泽，以及通过求解一组局部未知区域的泊松方程来改善无光泽。 还提出了各种其它示例性技术。

公开(公告)号：US09064476B2

公开(公告)日：2015-06-23

申请号：US12245712

申请日：2008-10-04

申请人： Jian Sun ,  Heung-Yeung Shum

发明人： Jian Sun ,  Heung-Yeung Shum

IPC分类号： G06K9/32 ,  G09G5/391 ,  G06T3/40

CPC分类号： G09G5/391 ,  G06T3/4053 ,  G09G2340/0407

摘要： Described is a technology by which a low-resolution image is processed into a high-resolution image, including by performing processing in the gradient domain. A gradient profile corresponding to the lower-resolution image is transform into a sharpened image gradient. A high-resolution gradient profile is estimated from a low-resolution gradient profile, e.g., by multiplying the low-resolution gradient profile by a transform ratio that is based upon learned shape parameters, learned sharpness values and a curve distance to an edge pixel along the gradient profile. The transform ratio is used to transform a low-resolution gradient field to a high-resolution gradient field. Reconstructing the higher-resolution image is performed by using the high-resolution gradient field as a gradient domain constraint, e.g., in along with a reconstruction constraint obtained from image domain data. An energy function is minimized by enforcing the gradient domain constraint and the reconstruction constraint, e.g., by performing a gradient descent algorithm.

摘要翻译： 描述了将低分辨率图像处理成高分辨率图像的技术，包括通过在梯度域中执行处理。 对应于较低分辨率图像的渐变曲线被转换成锐化的图像梯度。 从低分辨率梯度轮廓估计高分辨率梯度轮廓，例如，通过将低分辨率梯度轮廓乘以基于学习的形状参数，学习的锐度值和到边缘像素的曲线距离的变换比， 梯度轮廓。 变换比用于将低分辨率梯度场转换为高分辨率梯度场。 通过使用高分辨率梯度场作为梯度域约束，例如与从图像域数据获得的重建约束一起，进行重建高分辨率图像。 通过执行梯度域约束和重构约束，例如通过执行梯度下降算法来最小化能量函数。

公开(公告)号：US07576755B2

公开(公告)日：2009-08-18

申请号：US11674243

申请日：2007-02-13

申请人： Jian Sun ,  Xiaoou Tang ,  Heung-Yeung Shum

发明人： Jian Sun ,  Xiaoou Tang ,  Heung-Yeung Shum

IPC分类号： G09G5/00

CPC分类号： G06T11/60

摘要： Systems and methods provide picture collage systems and methods. In one implementation, a system determines a salient region in each of multiple images and develops a Bayesian model to maximize visibility of the salient regions in a collage that overlaps the images. The Bayesian model can also minimize blank spaces in the collage and normalize the percentage of each salient region that can be visibly displayed in the collage. Images are placed with diversified rotational orientation to provide a natural artistic collage appearance. A Markov Chain Monte Carlo technique is applied to the parameters of the Bayesian model to obtain image placement, orientation, and layering. The MCMC technique can combine optimization proposals that include local, global, and pairwise samplings from a distribution of state variables.

摘要翻译： 系统和方法提供图片拼贴系统和方法。 在一个实现中，系统确定多个图像中的每一个中的显着区域，并且开发贝叶斯模型以最大化与图像重叠的拼贴中的显着区域的可见性。 贝叶斯模型还可以将拼贴中的空白空间最小化，并将每个显着区域的百分比归一化，可以在拼贴画中显示。 图像以多样化的旋转方向放置，以提供自然的艺术拼贴外观。 将马尔科夫链蒙特卡罗技术应用于贝叶斯模型的参数，以获得图像放置，取向和分层。 MCMC技术可以结合来自状态变量分布的本地，全局和成对采样的优化提议。

公开(公告)号：US20070216675A1

公开(公告)日：2007-09-20

申请号：US11467859

申请日：2006-08-28

申请人： Jian Sun ,  Qiang Wang ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

发明人： Jian Sun ,  Qiang Wang ,  Weiwei Zhang ,  Xiaoou Tang ,  Heung-Yeung Shum

IPC分类号： H04N13/04 ,  G06T15/00

CPC分类号： G06T11/00

摘要： Digital video effects are described. In one aspect, a foreground object in a video stream is identified. The video stream comprises multiple image frames. The foreground object is modified by rendering a 3-dimensional (3-D) visual feature over the foreground object for presentation to a user in a modified video stream. Pose of the foreground object is tracked in 3-D space across respective ones of the image frames to identify when the foreground object changes position in respective ones of the image frames. Based on this pose tracking, aspect ratio of the 3-D visual feature is adaptively modified and rendered over the foreground object in corresponding image frames for presentation to the user in the modified video stream.

摘要翻译： 描述数字视频效果。 在一个方面，识别视频流中的前景对象。 视频流包括多个图像帧。 通过在前景对象上呈现三维（3-D）视觉特征来修改前景对象，以呈现给经修改的视频流中的用户。 前景物体的姿态在相应的图像帧中的3-D空间中被跟​​踪，以识别前景对象何时改变相应图像帧中的位置。 基于这种姿态跟踪，3-D视觉特征的宽高比被自适应地修改并在相应图像帧中的前景对象上呈现，以便在修改的视频流中呈现给用户。

公开(公告)号：US20060285762A1

公开(公告)日：2006-12-21

申请号：US11158113

申请日：2005-06-21

申请人： Jian Sun ,  Heung-Yeung Shum

发明人： Jian Sun ,  Heung-Yeung Shum

IPC分类号： G06K9/40

CPC分类号： G06T5/005

摘要： Image completion with structure propagation is described. In one aspect, synthesized patches for an unknown region in an input image are automatically generated. The synthesized patches are generated from a known region in the input image based on information from one or more curves. The one or more curves were generated to provide missing structure to the unknown region. Structure is propagated to the unknown region with the synthesized patches.

公开(公告)号：US20050219264A1

公开(公告)日：2005-10-06

申请号：US10817516

申请日：2004-04-01

申请人： Heung-Yeung Shum ,  Jian Sun ,  Shuntaro Yamazaki ,  Yin Li

发明人： Heung-Yeung Shum ,  Jian Sun ,  Shuntaro Yamazaki ,  Yin Li

IPC分类号： G06K9/00 ,  G06T15/20 ,  G09G5/00

CPC分类号： G06T15/205

摘要： Techniques are disclosed to produce virtual views of a complex scene. The virtual views are substantially free from aliasing even when using a relatively sparse set of images of the scene. In a described implementation, a scene is split into one or more coherent layers. The boundaries of the coherent layers are propagated across a plurality of frames corresponding to the scene. The splitting may be further refined (e.g., in accordance with user feedback) to present a virtual view of the scene.

摘要翻译： 公开了产生复杂场景的虚拟视图的技术。 即使当使用场景的相对稀疏的图像集时，虚拟视图也基本上没有混叠。 在所描述的实现中，场景被分成一个或多个相干层。 相干层的边界跨越对应于场景的多个帧传播。 可以进一步改进分割（例如，根据用户反馈）来呈现场景的虚拟视图。

公开(公告)号：US08351713B2

公开(公告)日：2013-01-08

申请号：US11676806

申请日：2007-02-20

申请人： Jian Sun ,  Jiaya Jia ,  Heung-Yeung Shum

发明人： Jian Sun ,  Jiaya Jia ,  Heung-Yeung Shum

IPC分类号： G06K9/36

CPC分类号： H04N1/3876 ,  G06T7/12 ,  G06T7/162 ,  G06T7/194 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20096

摘要： Systems and methods provide drag-and-drop pasting for seamless image composition. In one implementation, a user casually outlines a region of a source image that contains a visual object to be pasted into a target image. An exemplary system automatically calculates a new boundary within this region, such that when pasted at this boundary, visual seams are minimized. The system applies a shortest path calculation to find the optimal pasting boundary. The best path has minimal color variation along its length, thus avoiding structure and visual objects in the target image and providing the best chance for seamlessness. Poisson image editing is applied across this optimized boundary to blend colors. When the visual object being pasted has fine structure at its border that could be truncated by the Poisson editing, the exemplary system integrates the alpha matte of the visual object into the Poisson equations to protect the fine structure.

摘要翻译： 系统和方法为无缝图像组合提供拖放粘贴。 在一个实现中，用户随意地概括了包含要粘贴到目标图像中的视觉对象的源图像的区域。 示例性系统自动计算该区域内的新边界，使得当粘贴在该边界处时，视觉接缝被最小化。 系统应用最短路径计算来找到最佳粘贴边界。 最佳路径沿其长度具有最小的颜色变化，从而避免了目标图像中的结构和视觉对象，并提供了无缝隙的最佳机会。 Poisson图像编辑应用于这个优化的边界以混合颜色。 当粘贴的视觉对象在其边界处具有可被泊松编辑截断的精细结构时，示例性系统将视觉对象的阿尔法无光泽整合到泊松方程中以保护精细结构。

公开(公告)号：US08238694B2

公开(公告)日：2012-08-07

申请号：US12245339

申请日：2008-10-03

申请人： Jian Sun ,  Heung-Yeung Shum

发明人： Jian Sun ,  Heung-Yeung Shum

IPC分类号： G06K9/32 ,  G06K9/40

CPC分类号： G06T7/35

摘要： The alignment of a sharp image of a subject and a blurred image of the same subject is disclosed. For example, one disclosed embodiment provides a method of determining a series of trial images. The method comprises applying a corresponding series of coordinate transforms to the sharp image, the series of coordinate transforms differing with respect to one or more of a rotational operation and a scaling operation. The method further comprises computing a series blur kernels corresponding to the series of trial images, each blur kernel mapping a trial image from the series of trial images to the blurred image. The method further includes locating a sparsest blur kernel in the series of blur kernels, and identifying one or more of the rotational operation and the scaling operation of the coordinate transform mapping the trial image corresponding to the sparsest blur kernel to the blurred image.

摘要翻译： 公开了对象的清晰图像与相同被摄体的模糊图像的对准。 例如，一个公开的实施例提供了确定一系列试验图像的方法。 该方法包括将相应的一系列坐标变换应用于锐利图像，所述一系列坐标变换相对于旋转操作和缩放操作中的一个或多个而不同。 该方法还包括计算与该系列试验图像相对应的一系列模糊粒子，每个模糊核心将来自一系列试验图像的试验图像映射到模糊图像。 该方法还包括将稀疏模糊核心定位在一系列模糊核心中，以及将映射与稀疏模糊核心的试用图像映射到模糊图像的坐标变换的旋转操作和缩放操作中的一个或多个。