公开(公告)号：US08019171B2

公开(公告)日：2011-09-13

申请号：US11736900

申请日：2007-04-18

申请人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

发明人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

IPC分类号： G06K9/36

CPC分类号： H04N19/12 ,  G06T7/12 ,  H04N19/17 ,  H04N19/61

摘要： Systems and methods provide vision-based image compression. In one implementation, inpainting is the vision-based technique selected to augment a conventional signal-processing-based technique. For some regions of a source image, an exemplary system efficiently extracts and organizes structural edge information instead of compressing the regions. In one implementation, the system applies binary curve fitting to capture the edge information. A structure-aware inpainter in the decoder can then restore the regions via the edge information, which occupies very little data space or minimal bandwidth in a bitstream that is transmitted from encoder to decoder. Key visual components of the image can still be conventionally compressed. Extracting edge information for some regions instead of compressing them considerably increases overall image compression.

摘要翻译： 系统和方法提供基于视觉的图像压缩。 在一个实现中，修复是基于视觉的技术，其被选择用于增加常规的基于信号处理的技术。 对于源图像的一些区域，示例性系统有效地提取并组织结构边缘信息，而不是压缩该区域。 在一个实现中，系统应用二进制曲线拟合来捕获边缘信息。 然后，解码器中的结构感知输入器可以经由边缘信息来恢复区域，该边缘信息在从编码器传送到解码器的比特流中占据非常小的数据空间或最小带宽。 图像的主要视觉部件仍然可以被传统地压缩。 为某些区域提取边缘信息而不是压缩边缘信息显着增加了整体图像压缩。

公开(公告)号：US08396312B2

公开(公告)日：2013-03-12

申请号：US13177434

申请日：2011-07-06

申请人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

发明人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

IPC分类号： G06K9/36

CPC分类号： H04N19/12 ,  G06T7/12 ,  H04N19/17 ,  H04N19/61

摘要： Systems and methods provide vision-based image compression. In one implementation, inpainting is the vision-based technique selected to augment a conventional signal-processing-based technique. For some regions of a source image, an exemplary system efficiently extracts and organizes structural edge information instead of compressing the regions. In one implementation, the system applies binary curve fitting to capture the edge information. A structure-aware inpainter in the decoder can then restore the regions via the edge information, which occupies very little data space or minimal bandwidth in a bitstream that is transmitted from encoder to decoder. Key visual components of the image can still be conventionally compressed. Extracting edge information for some regions instead of compressing them considerably increases overall image compression.

摘要翻译： 系统和方法提供基于视觉的图像压缩。 在一个实现中，修复是基于视觉的技术，其被选择用于增加常规的基于信号处理的技术。 对于源图像的一些区域，示例性系统有效地提取并组织结构边缘信息，而不是压缩该区域。 在一个实现中，系统应用二进制曲线拟合来捕获边缘信息。 然后，解码器中的结构感知输入器可以经由边缘信息来恢复区域，该边缘信息在从编码器传送到解码器的比特流中占据非常小的数据空间或最小带宽。 图像的主要视觉部件仍然可以被传统地压缩。 为某些区域提取边缘信息而不是压缩边缘信息显着增加了整体图像压缩。

公开(公告)号：US07755645B2

公开(公告)日：2010-07-13

申请号：US11615537

申请日：2007-03-29

申请人： Xiaoyan Sun ,  Dong Liu ,  Feng Wu ,  Shipeng Li

发明人： Xiaoyan Sun ,  Dong Liu ,  Feng Wu ,  Shipeng Li

IPC分类号： G09G5/00

CPC分类号： G06T11/60 ,  G06T5/005

摘要： Systems and methods provide object-based image inpainting. The object-based inpainting can recover part of a missing object or can synthesize visual objects that are entirely missing. In one implementation, an exemplary system generates a target visual object to be inpainted into an image, by geometrically matching key features with a model visual object that can be selected from the same or another image. Borders of the target object are refined through pixel-based contour deformation and textures for the target object are obtained either by pixel interpolation or by patch-based modeling of pre-existing texture in the image. Graph-cut segmentation rectifies image overlap as the target object is embedded in the image. Poisson editing filters visual seams, and edge filling completes visual gaps around the border of the target object.

摘要翻译： 系统和方法提供基于对象的图像修复。 基于对象的修复可以恢复缺少对象的一部分，或者可以合成完全缺失的可视对象。 在一个实现中，示例性系统通过将关键特征与可以从相同图像或另一图像中选择的模型视觉对象几何匹配来生成要修饰成图像的目标视觉对象。 通过基于像素的轮廓变形来改进目标对象的边界，通过像素插值或通过图像中预先存在的纹理的基于块的建模来获得目标对象的纹理。 当目标物体嵌入在图像中时，图形分割会整理图像重叠。 泊松编辑过滤视觉接缝，边缘填充完成目标物体边界周围的视觉间隙。

公开(公告)号：US20080238942A1

公开(公告)日：2008-10-02

申请号：US11615537

申请日：2007-03-29

申请人： Xiaoyan Sun ,  Dong Liu ,  Feng Wu ,  Shipeng Li

发明人： Xiaoyan Sun ,  Dong Liu ,  Feng Wu ,  Shipeng Li

IPC分类号： G09G5/00

CPC分类号： G06T11/60 ,  G06T5/005

摘要： Systems and methods provide object-based image inpainting. The object-based inpainting can recover part of a missing object or can synthesize visual objects that are entirely missing. In one implementation, an exemplary system generates a target visual object to be inpainted into an image, by geometrically matching key features with a model visual object that can be selected from the same or another image. Borders of the target object are refined through pixel-based contour deformation and textures for the target object are obtained either by pixel interpolation or by patch-based modeling of pre-existing texture in the image. Graph-cut segmentation rectifies image overlap as the target object is embedded in the image. Poisson editing filters visual seams, and edge filling completes visual gaps around the border of the target object.

摘要翻译： 系统和方法提供基于对象的图像修复。 基于对象的修复可以恢复缺少对象的一部分，或者可以合成完全缺失的可视对象。 在一个实现中，示例性系统通过将关键特征与可以从相同图像或另一图像中选择的模型视觉对象几何匹配来生成要修饰成图像的目标视觉对象。 通过基于像素的轮廓变形来改进目标对象的边界，通过像素插值或通过图像中预先存在的纹理的基于块的建模来获得目标对象的纹理。 当目标物体嵌入在图像中时，图形分割会整理图像重叠。 泊松编辑过滤视觉接缝，边缘填充完成目标物体边界周围的视觉间隙。

公开(公告)号：US20110262049A1

公开(公告)日：2011-10-27

申请号：US13177434

申请日：2011-07-06

申请人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

发明人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

IPC分类号： G06K9/36

CPC分类号： H04N19/12 ,  G06T7/12 ,  H04N19/17 ,  H04N19/61

摘要： Systems and methods provide vision-based image compression. In one implementation, inpainting is the vision-based technique selected to augment a conventional signal-processing-based technique. For some regions of a source image, an exemplary system efficiently extracts and organizes structural edge information instead of compressing the regions. In one implementation, the system applies binary curve fitting to capture the edge information. A structure-aware inpainter in the decoder can then restore the regions via the edge information, which occupies very little data space or minimal bandwidth in a bitstream that is transmitted from encoder to decoder. Key visual components of the image can still be conventionally compressed. Extracting edge information for some regions instead of compressing them considerably increases overall image compression.

摘要翻译： 系统和方法提供基于视觉的图像压缩。 在一个实现中，修复是基于视觉的技术，其被选择用于增加常规的基于信号处理的技术。 对于源图像的一些区域，示例性系统有效地提取并组织结构边缘信息，而不是压缩该区域。 在一个实现中，系统应用二进制曲线拟合来捕获边缘信息。 然后，解码器中的结构感知输入器可以经由边缘信息来恢复区域，该边缘信息在从编码器传送到解码器的比特流中占据非常小的数据空间或最小带宽。 图像的主要视觉部件仍然可以被传统地压缩。 为某些区域提取边缘信息而不是压缩边缘信息显着增加了整体图像压缩。

公开(公告)号：US20070248272A1

公开(公告)日：2007-10-25

申请号：US11736900

申请日：2007-04-18

申请人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

发明人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li ,  Dong Liu

IPC分类号： G06K9/36

CPC分类号： H04N19/12 ,  G06T7/12 ,  H04N19/17 ,  H04N19/61

摘要： Systems and methods provide vision-based image compression. In one implementation, inpainting is the vision-based technique selected to augment a conventional signal-processing-based technique. For some regions of a source image, an exemplary system efficiently extracts and organizes structural edge information instead of compressing the regions. In one implementation, the system applies binary curve fitting to capture the edge information. A structure-aware inpainter in the decoder can then restore the regions via the edge information, which occupies very little data space or minimal bandwidth in a bitstream that is transmitted from encoder to decoder. Key visual components of the image can still be conventionally compressed. Extracting edge information for some regions instead of compressing them considerably increases overall image compression.

摘要翻译： 系统和方法提供基于视觉的图像压缩。 在一个实现中，修复是基于视觉的技术，其被选择用于增加常规的基于信号处理的技术。 对于源图像的一些区域，示例性系统有效地提取并组织结构边缘信息，而不是压缩该区域。 在一个实现中，系统应用二进制曲线拟合来捕获边缘信息。 然后，解码器中的结构感知输入器可以经由边缘信息来恢复区域，该边缘信息在从编码器传送到解码器的比特流中占据非常小的数据空间或最小带宽。 图像的主要视觉部件仍然可以被传统地压缩。 为某些区域提取边缘信息而不是压缩边缘信息显着增加了整体图像压缩。

公开(公告)号：US08208556B2

公开(公告)日：2012-06-26

申请号：US11768862

申请日：2007-06-26

申请人： Xiaoyan Sun ,  Chunbo Zhu ,  Feng Wu ,  Shipeng Li

发明人： Xiaoyan Sun ,  Chunbo Zhu ,  Feng Wu ,  Shipeng Li

IPC分类号： H04N11/04

CPC分类号： G06T7/40 ,  G06T2207/10016 ,  H04N19/27 ,  H04N19/577

摘要： Systems and methods for video coding using spatio-temporal texture synthesis are described. In one aspect, a video data coding pipeline portion of the codec removes texture blocks from the video data to generate coded video data. The removed texture blocks are selected based on an objective determination that each of the remove texture blocks can be synthesized from spatio-temporal neighboring samples during decoding operations. The objective determinations are made using local block-based motion information independent of global motion models. An indication of which texture blocks were removed is provided to a decoder in addition to the coded video data. Decoding logic of the codec decodes the video data using a standard decoding algorithm. The decoding logic also restores the removed texture blocks via spatio-temporal texture synthesis to generate synthesized video data. The decoded and synthesized video data is presented to a user.

摘要翻译： 描述使用时空纹理合成的视频编码的系统和方法。 一方面，编解码器的视频数据编码流水线部分从视频数据中去除纹理块以产生编码视频数据。 基于在解码操作期间可以从空时相邻采样中合成每个去除纹理块的目标确定来选择去除的纹理块。 使用与全局运动模型无关的局部基于块的运动信息进行客观确定。 去除了纹理块的指示除了编码的视频数据之外还提供给解码器。 编解码器的解码逻辑使用标准解码算法解码视频数据。 解码逻辑还通过空间 - 时间纹理合成恢复去除的纹理块，以产生合成的视频数据。 解码和合成的视频数据被呈现给用户。

公开(公告)号：US07583730B2

公开(公告)日：2009-09-01

申请号：US11070905

申请日：2005-03-03

申请人： Feng Wu ,  Shipeng Li ,  Ya-Qin Zhang ,  Bing Zeng ,  Xiaoyan Sun

发明人： Feng Wu ,  Shipeng Li ,  Ya-Qin Zhang ,  Bing Zeng ,  Xiaoyan Sun

IPC分类号： H04N7/12

CPC分类号： H04N19/34 ,  H04N19/51

摘要： A motion-compensated video encoding scheme employs progressive fine-granularity layered coding to encode macroblocks of video data into frames having multiple layers, including a base layer of comparatively low quality video and multiple enhancement layers of increasingly higher quality video. Some of the enhancement layers in a current frame are predicted from different quality layers in reference frames. The video encoding scheme estimates drifting errors during the encoding and chooses a coding mode for each macroblock in the enhancement layer to maximize high coding efficiency while minimizing drifting errors.

公开(公告)号：US08311347B2

公开(公告)日：2012-11-13

申请号：US11558755

申请日：2006-11-10

申请人： Xiaoyan Sun ,  Feng Wu ,  Zhiwei Xiong ,  Shipeng Li

发明人： Xiaoyan Sun ,  Feng Wu ,  Zhiwei Xiong ,  Shipeng Li

IPC分类号： G06K9/36 ,  G06K9/46

CPC分类号： H04N19/17 ,  G06K9/0008 ,  G06K9/00744 ,  G06T9/00 ,  H04N19/134

摘要： Systems and methods provide image compression based on parameter-assisted inpainting. In one implementation of an encoder, an image is partitioned into blocks and the blocks classified as smooth or unsmooth, based on the degree of visual edge content and chromatic variation in each block. Image content of the unsmooth blocks is compressed, while image content of the smooth blocks is summarized by parameters, but not compressed. The parameters, once obtained, may also be compressed. At a decoder, the compressed image content of the unsmooth blocks and the compressed parameters of the smooth blocks are each decompressed. Each smooth block is then reconstructed by inpainting, guided by the parameters in order to impart visual detail from the original image that cannot be implied from the image content of neighboring blocks that have been decoded.

摘要翻译： 系统和方法提供基于参数辅助修复的图像压缩。 在编码器的一个实现中，基于每个块中的视觉边缘内容的程度和色度变化，将图像划分为块，并将块分类为平滑或不平滑。 不平滑块的图像内容被压缩，而平滑块的图像内容由参数汇总，但不被压缩。 一旦获得的参数也可以被压缩。 在解码器处，解压缩不平滑块的压缩图像内容和平滑块的压缩参数。 然后通过修饰重建每个平滑块，由参数引导，以便从原始图像传递不能从已经被解码的相邻块的图像内容中隐含的视觉细节。

公开(公告)号：US20090067491A1

公开(公告)日：2009-03-12

申请号：US11851653

申请日：2007-09-07

申请人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li

发明人： Xiaoyan Sun ,  Feng Wu ,  Shipeng Li

IPC分类号： G06T9/00

CPC分类号： H04N19/59 ,  H04N19/132 ,  H04N19/14 ,  H04N19/587 ,  H04N19/85 ,  H04N19/90

摘要： Learning-based image compression is described. In one implementation, an encoder possessing a first set of learned visual knowledge primitives excludes visual information from an image prior to compression. A decoder possessing an independently learned set of visual knowledge primitives synthesizes the excluded visual information into the image after decompression. The encoder and decoder are decoupled with respect to the information excluded at the encoder and the information synthesized at the decoder. This results in superior data compression since the information excluded at the encoder is dropped completely and not transferred to the decoder. Primitive visual elements synthesized at the decoder may be different than primitive visual elements dropped at the encoder, but the resulting reconstituted image is perceptually equivalent to the original image.

摘要翻译： 描述基于学习的图像压缩。 在一个实现中，具有第一组学习视觉知识原语的编码器在压缩之前从图像中排除视觉信息。 具有独立学习的视觉知识图元组的解码器在解压缩之后将排除的视觉信息合成到图像中。 编码器和解码器相对于在编码器处排除的信息和在解码器处合成的信息去耦合。 这导致优异的数据压缩，因为在编码器处排除的信息完全丢弃并且不传送到解码器。 在解码器处合成的原始视觉元素可能不同于在编码器处丢弃的原始视觉元素，但是所产生的重构图像在听觉上等同于原始图像。