公开(公告)号：US07477802B2

公开(公告)日：2009-01-13

申请号：US11601518

申请日：2006-11-16

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad ,  Michael D. Robinson

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad ,  Michael D. Robinson

IPC分类号： G06K9/32

CPC分类号： G06T3/4053

摘要： A computer method of creating a super-resolved grayscale image from lower-resolution images using an L1 norm data fidelity penalty term to enforce similarities between low and a high-resolution image estimates is provided. A spatial penalty term encourages sharp edges in the high-resolution image, the data fidelity penalty term is applied to space invariant point spread function, translational, affine, projective and dense motion models including fusing the lower-resolution images, to estimate a blurred higher-resolution image and then a deblurred image. The data fidelity penalty term uses the L1 norm in a likelihood fidelity term for motion estimation errors. The spatial penalty term uses bilateral-TV regularization with an image having horizontal and vertical pixel-shift terms, and a scalar weight between 0 and 1. The penalty terms create an overall cost function having steepest descent optimization applied for minimization. Direct image operator effects replace matrices for speed and efficiency.

摘要翻译： 提供了一种使用L1范数数据保真度惩罚项从低分辨率图像创建超分辨灰度图像以实现低分辨率图像估计和高分辨率图像估计之间的相似性的计算机方法。 空间惩罚项鼓励高分辨率图像中的尖锐边缘，将数据保真度惩罚项应用于空间不变点扩散函数，平移，仿射，投射和密集运动模型，包括融合较低分辨率图像，估计模糊较高 分辨率图像，然后去除图像。 数据保真度惩罚项在运动估计误差的似然保真项中使用L1范数。 空间惩罚项使用具有水平和垂直像素移位项的图像的双向TV正则化，以及0到1之间的标量权重。惩罚项创建具有最小下降最优化的总体成本函数。 直接图像运算符效应代替矩阵的速度和效率。

公开(公告)号：US07412107B2

公开(公告)日：2008-08-12

申请号：US11301811

申请日：2005-12-12

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

IPC分类号： G06K9/40

CPC分类号： G06T3/4015 ,  G06T5/002 ,  G06T5/003 ,  G06T2207/10024 ,  G06T2207/20192

摘要： An integrated method for both super-resolution and multi-frame demosaicing includes an image fusion followed by simultaneous deblurring and interpolation. For the case of color super-resolution, the first step involves application of recursive image fusion separately on the three different color layers. The second step is based on minimizing a maximum a posteriori (MAP) cost function. In one embodiment, the MAP cost function is composed of several terms: a data fidelity penalty term that penalizes dissimilarity between the raw data and the super-resolved estimate, a luminance penalty term that favors sharp edges in the luminance component of the image, a chrominance penalty term that favors low spatial frequency changes in the chrominance component of the image, and an orientation penalty term that favors similar edge orientations across the color channels. The method is also applicable to color super-resolution (without demosaicing), where the low-quality input images are already demosaiced. In addition, for translational motion, the method may be used in a very fast image fusion algorithm to facilitate the implementation of dynamic, multi-input/multi-output color super-resolution/demosaicing.

摘要翻译： 用于超分辨率和多帧去马赛克的集成方法包括图像融合，随后同时去模糊和插值。 对于彩色超分辨率的情况，第一步涉及分别在三个不同颜色层上应用递归图像融合。 第二步是基于最大化后验（MAP）成本函数。 在一个实施例中，MAP成本函数由几个术语组成：惩罚原始数据与超分辨估计之间的不相似性的数据保真度惩罚项，有利于图像的亮度分量中的锐利边缘的亮度惩罚项， 有利于图像的色度分量中的低空间频率变化的色度惩罚项，以及有利于穿过颜色通道的相似边缘方向的取向惩罚项。 该方法也适用于低质量输入图像已经被去马赛克的颜色超分辨率（无去马赛克）。 此外，对于平移运动，该方法可以用于非常快速的图像融合算法，以便于动态，多输入/多输出颜色超分辨率/去马赛克的实现。

公开(公告)号：US20060291750A1

公开(公告)日：2006-12-28

申请号：US11301817

申请日：2005-12-12

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

IPC分类号： G06K9/32

CPC分类号： G06T3/4076 ,  G06K9/20

摘要： In one aspect, the present invention provides a dynamic super-resolution technique that is computationally efficient. A recursive computation takes as input a previously computed super-resolved image derived from a sequence of low-resolution input frames. Combining this super-resolved image with a later low-resolution input frame in the sequence, the technique produces a new super-resolved image. By recursive application, a sequence of super-resolved images is produced. In a preferred embodiment, the technique uses a computationally simple and effective method based on adaptive filtering for computing a high resolution image and updating this high resolution image over time to produce an enhanced sequence of images. The method may be implemented as a general super-resolution software tool capable of handing a wide variety of input image data.

摘要翻译： 一方面，本发明提供了一种计算效率高的动态超分辨率技术。 递归计算作为从低分辨率输入帧序列导出的先前计算的超分辨率图像的输入。 将该超分辨率图像与序列中的较低分辨率输入帧组合，该技术产生新的超分辨图像。 通过递归应用，产生一系列超分辨率图像。 在优选实施例中，该技术使用基于自适应滤波的计算简单且有效的方法来计算高分辨率图像并随时间更新该高分辨率图像以产生增强的图像序列。 该方法可以被实现为能够处理各种各样的输入图像数据的一般超分辨率软件工具。

公开(公告)号：US20060291751A1

公开(公告)日：2006-12-28

申请号：US11302073

申请日：2005-12-12

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad ,  Michael Robinson

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad ,  Michael Robinson

IPC分类号： G06K9/32

CPC分类号： G06T3/4053

摘要： A method for computing a high resolution gray-tone image from a sequence of low-resolution images uses an L1 norm minimization. In a preferred embodiment, the technique also uses a robust regularization based on a bilateral prior to deal with different data and noise models. This robust super-resolution technique uses the L1 norm both for the regularization and the data fusion terms. Whereas the former is responsible for edge preservation, the latter seeks robustness with respect to motion error, blur, outliers, and other kinds of errors not explicitly modeled in the fused images. This computationally inexpensive method is resilient against errors in motion and blur estimation, resulting in images with sharp edges. The method also reduces the effects of aliasing, noise and compression artifacts. The method's performance is superior to other super-resolution methods and has fast convergence.

摘要翻译： 用于从低分辨率图像序列计算高分辨率灰度色调图像的方法使用L 1范数最小化。 在优选实施例中，该技术还在处理不同数据和噪声模型之前基于双边的鲁棒正则化。 这种强大的超分辨率技术对于正则化和数据融合项都使用L 1标准。 而前者负责边缘保护，后者针对运动误差，模糊，异常值和融合图像中没有明确建模的其他类型的错误寻求鲁棒性。 这种计算上便宜的方法对于运动和模糊估计中的错误具有弹性，导致具有锐利边缘的图像。 该方法还减少了混叠，噪声和压缩伪影的影响。 该方法的性能优于其他超分辨率方法，并具有快速收敛性。

公开(公告)号：US20060290711A1

公开(公告)日：2006-12-28

申请号：US11301811

申请日：2005-12-12

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

IPC分类号： G09G5/00

CPC分类号： G06T3/4015 ,  G06T5/002 ,  G06T5/003 ,  G06T2207/10024 ,  G06T2207/20192

摘要： An integrated method for both super-resolution and multi-frame demosaicing includes an image fusion followed by simultaneous deblurring and interpolation. For the case of color super-resolution, the first step involves application of recursive image fusion separately on the three different color layers. The second step is based on minimizing a maximum a posteriori (MAP) cost function. In one embodiment, the MAP cost function is composed of several terms: a data fidelity penalty term that penalizes dissimilarity between the raw data and the super-resolved estimate, a luminance penalty term that favors sharp edges in the luminance component of the image, a chrominance penalty term that favors low spatial frequency changes in the chrominance component of the image, and an orientation penalty term that favors similar edge orientations across the color channels. The method is also applicable to color super-resolution (without demosaicing), where the low-quality input images are already demosaiced. In addition, for translational motion, the method may be used in a very fast image fusion algorithm to facilitate the implementation of dynamic, multi-input/multi-output color super-resolution/demosaicing.

摘要翻译： 用于超分辨率和多帧去马赛克的集成方法包括图像融合，随后同时去模糊和插值。 对于彩色超分辨率的情况，第一步涉及分别在三个不同颜色层上应用递归图像融合。 第二步是基于最大化后验（MAP）成本函数。 在一个实施例中，MAP成本函数由几个术语组成：惩罚原始数据与超分辨估计之间的不相似性的数据保真度惩罚项，有利于图像的亮度分量中的锐利边缘的亮度惩罚项， 有利于图像的色度分量中的低空间频率变化的色度惩罚项，以及有利于穿过颜色通道的相似边缘方向的取向惩罚项。 该方法也适用于低质量输入图像已经被去马赛克的颜色超分辨率（无去马赛克）。 此外，对于平移运动，该方法可以用于非常快速的图像融合算法，以便于动态，多输入/多输出颜色超分辨率/去马赛克的实现。

公开(公告)号：US20060279585A1

公开(公告)日：2006-12-14

申请号：US11506246

申请日：2006-08-17

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

IPC分类号： G09G5/02

CPC分类号： G06T3/4015 ,  G06T5/002 ,  G06T5/003 ,  G06T2207/10024 ,  G06T2207/20192

摘要： A method of creating a super-resolved color image from multiple lower-resolution color images is provided by combining a data fidelity penalty term, a spatial luminance penalty term, a spatial chrominance penalty term, and an inter-color dependencies penalty term to create an overall cost function. The data fidelity penalty term is an L1 norm penalty term to enforce similarities between raw data and a high-resolution image estimate, the spatial luminance penalty term is to encourage sharp edges in a luminance component to the high-resolution image, the spatial chrominance penalty term is to encourage smoothness in a chrominance component of the high-resolution image, and the inter-color dependencies penalty term is to encourage homogeneity of an edge location and orientation in different color bands. A steepest descent optimization is applied to the overall cost function for minimization by applying a derivative to each color band while the other color bands constant.

摘要翻译： 通过组合数据保真度罚分项，空间亮度惩罚项，空间色度惩罚项和颜色间依赖性惩罚项来提供从多个低分辨率彩色图像创建超分辨彩色图像的方法，以创建 整体成本函数。 数据保真度罚分项是用于强制原始数据与高分辨率图像估计之间的相似性的L1范数罚分项，空间亮度惩罚项是为了鼓励高分辨率图像的亮度分量中的锐利边缘，空间色度惩罚 术语是鼓励高分辨率图像的色度分量中的平滑度，并且颜色间依赖性惩罚项是为了促进不同颜色带中的边缘位置和取向的均匀性。 通过对每个色带应用导数，而另一个色带恒定，最快下降优化被应用于最小化的总体成本函数。

公开(公告)号：US07379612B2

公开(公告)日：2008-05-27

申请号：US11584400

申请日：2006-10-19

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

IPC分类号： G06K9/40

CPC分类号： G06T3/4076 ,  G06K9/20

摘要： A method is provided of solving the dynamic super-resolution (SR) problem of reconstructing a high-quality set of monochromatic or color superresolved images from low-quality monochromatic, color, or mosaiced frames. The invention includes a joint method for simultaneous SR, deblurring, and demosaicing, this way taking into account practical color measurements encountered in video sequences. For the case of translational motion and common space-invariant blur, the proposed invention is based on a very fast and memory efficient approximation of the Kalman filter (KF). Experimental results on both simulated and real data are supplied, demonstrating the invention algorithms, and their strength.

摘要翻译： 提供了一种解决从低质量单色，彩色或镶嵌帧重构高质量单色或彩色超解析图像的动态超分辨率（SR）问题的方法。 本发明包括用于同时进行SR，去模糊和去马赛克的联合方法，这种方式考虑了视频序列中遇到的实际颜色测量。 对于平移运动和公共空间不变模糊的情况，所提出的发明基于卡尔曼滤波器（KF）的非常快速且高记忆效率的近似。 提供了模拟和实际数据的实验结果，展示了本发明的算法及其实力。

公开(公告)号：US20070071362A1

公开(公告)日：2007-03-29

申请号：US11584400

申请日：2006-10-19

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

IPC分类号： G06K9/32

CPC分类号： G06T3/4076 ,  G06K9/20

摘要： A method is provided of solving the dynamic super-resolution (SR) problem of reconstructing a high-quality set of monochromatic or color superresolved images from low-quality monochromatic, color, or mosaiced frames. The invention includes a joint method for simultaneous SR, deblurring, and demosaicing, this way taking into account practical color measurements encountered in video sequences. For the case of translational motion and common space-invariant blur, the proposed invention is based on a very fast and memory efficient approximation of the Kalman filter (KF). Experimental results on both simulated and real data are supplied, demonstrating the invention algorithms, and their strength.

摘要翻译： 提供了一种解决从低质量单色，彩色或镶嵌帧重构高质量单色或彩色超解析图像的动态超分辨率（SR）问题的方法。 本发明包括用于同时进行SR，去模糊和去马赛克的联合方法，这种方式考虑了视频序列中遇到的实际颜色测量。 对于平移运动和公共空间不变模糊的情况，所提出的发明基于卡尔曼滤波器（KF）的非常快速且高记忆效率的近似。 提供了模拟和实际数据的实验结果，展示了本发明的算法及其实力。

公开(公告)号：US07940282B2

公开(公告)日：2011-05-10

申请号：US11506246

申请日：2006-08-17

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad

IPC分类号： G09G5/00 ,  G09G5/02 ,  H04N9/04 ,  H04N5/00 ,  H03L7/00 ,  G03F3/08 ,  G06K9/00 ,  H04N1/46 ,  G06K9/40 ,  G06K9/32

CPC分类号： G06T3/4015 ,  G06T5/002 ,  G06T5/003 ,  G06T2207/10024 ,  G06T2207/20192

摘要： A method of creating a super-resolved color image from multiple lower-resolution color images is provided by combining a data fidelity penalty term, a spatial luminance penalty term, a spatial chrominance penalty term, and an inter-color dependencies penalty term to create an overall cost function. The data fidelity penalty term is an L1 norm penalty term to enforce similarities between raw data and a high-resolution image estimate, the spatial luminance penalty term is to encourage sharp edges in a luminance component to the high-resolution image, the spatial chrominance penalty term is to encourage smoothness in a chrominance component of the high-resolution image, and the inter-color dependencies penalty term is to encourage homogeneity of an edge location and orientation in different color bands. A steepest descent optimization is applied to the overall cost function for minimization by applying a derivative to each color band while the other color bands constant.

摘要翻译： 通过组合数据保真度罚分项，空间亮度惩罚项，空间色度惩罚项和颜色间依赖性惩罚项来提供从多个低分辨率彩色图像创建超分辨彩色图像的方法，以创建 整体成本函数。 数据保真度罚分项是用于强制原始数据与高分辨率图像估计之间的相似性的L1范数罚分项，空间亮度惩罚项是为了鼓励高分辨率图像的亮度分量中的锐利边缘，空间色度惩罚 术语是鼓励高分辨率图像的色度分量中的平滑度，并且颜色间依赖性惩罚项是为了促进不同颜色带中的边缘位置和取向的均匀性。 通过对每个色带应用导数，而另一个色带恒定，最快下降优化被应用于最小化的总体成本函数。

公开(公告)号：US20070217713A1

公开(公告)日：2007-09-20

申请号：US11601518

申请日：2006-11-16

申请人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad ,  Michael Robinson

发明人： Peyman Milanfar ,  Sina Farsiu ,  Michael Elad ,  Michael Robinson

IPC分类号： G06K9/32

CPC分类号： G06T3/4053

摘要： A computer method of creating a super-resolved grayscale image from lower-resolution images using an L1 norm data fidelity penalty term to enforce similarities between low and a high-resolution image estimates is provided. A spatial penalty term encourages sharp edges in the high-resolution image, the data fidelity penalty term is applied to space invariant point spread function, translational, affine, projective and dense motion models including fusing the lower-resolution images, to estimate a blurred higher-resolution image and then a deblurred image. The data fidelity penalty term uses the L1 norm in a likelihood fidelity term for motion estimation errors. The spatial penalty term uses bilateral-TV regularization with an image having horizontal and vertical pixel-shift terms, and a scalar weight between 0 and 1. The penalty terms create an overall cost function having steepest descent optimization applied for minimization. Direct image operator effects replace matrices for speed and efficiency.

摘要翻译： 提供了一种使用L 1范数数据保真度罚分项从低分辨率图像创建超分辨灰度图像以实现低分辨率图像估计和高分辨率图像估计之间的相似性的计算机方法。 空间惩罚项鼓励高分辨率图像中的尖锐边缘，将数据保真度惩罚项应用于空间不变点扩散函数，平移，仿射，投影和密集运动模型，包括融合较低分辨率图像，估计模糊较高 分辨率图像，然后去除图像。 数据保真度惩罚项在运动估计误差的似然保真度项中使用L 1 1范数。 空间惩罚项使用具有水平和垂直像素移位项的图像的双向TV正则化，以及0到1之间的标量权重。惩罚项创建具有最小下降最优化的总成本函数。 直接图像运算符效应代替矩阵的速度和效率。