公开(公告)号：US20120147973A1

公开(公告)日：2012-06-14

申请号：US12966914

申请日：2010-12-13

申请人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani

发明人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani

IPC分类号： H04N7/26

CPC分类号： H04N19/188 ,  H04N19/103 ,  H04N19/156 ,  H04N19/1883 ,  H04N19/42 ,  H04N19/44 ,  H04N19/70

摘要： Techniques and tools for reducing latency in video decoding for real-time communication applications that emphasize low delay. For example, a tool such as a video decoder selects a low-latency decoding mode. Based on the selected decoding mode, the tool adjusts output timing determination, picture boundary detection, number of pictures in flight and/or jitter buffer utilization. For low-latency decoding, the tool can use a frame count syntax element to set initial output delay for a decoded picture buffer, and the tool can use auxiliary delimiter syntax elements to detect picture boundaries. To further reduce delay in low-latency decoding, the tool can reduce number of pictures in flight for multi-threaded decoding and reduce or remove jitter buffers. The tool receives encoded data, performs decoding according to the selected decoding mode to reconstruct pictures, and outputs the pictures for display.

摘要翻译： 用于减少视频解码中延迟延迟的技术和工具，用于强调低延迟的实时通信应用。 例如，诸如视频解码器的工具选择低延迟解码模式。 基于选择的解码模式，该工具调整输出定时确定，图像边界检测，飞行中的图像数和/或抖动缓冲器利用率。 对于低延迟解码，该工具可以使用帧计数语法元素为解码图像缓冲区设置初始输出延迟，并且该工具可以使用辅助分隔符语法元素来检测图像边界。 为了进一步减少低延迟解码的延迟，该工具可以减少飞行中用于多线程解码的图片数量，并减少或删除抖动缓冲区。 该工具接收编码数据，根据选择的解码模式执行解码以重构图像，并输出用于显示的图像。

公开(公告)号：US08885729B2

公开(公告)日：2014-11-11

申请号：US12966914

申请日：2010-12-13

申请人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani

发明人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani

IPC分类号： H04N11/02 ,  H04N19/103 ,  H04N19/44 ,  H04N19/42 ,  H04N19/70 ,  H04N19/156

CPC分类号： H04N19/188 ,  H04N19/103 ,  H04N19/156 ,  H04N19/1883 ,  H04N19/42 ,  H04N19/44 ,  H04N19/70

摘要： Techniques and tools for reducing latency in video decoding for real-time communication applications that emphasize low delay. For example, a tool such as a video decoder selects a low-latency decoding mode. Based on the selected decoding mode, the tool adjusts output timing determination, picture boundary detection, number of pictures in flight and/or jitter buffer utilization. For low-latency decoding, the tool can use a frame count syntax element to set initial output delay for a decoded picture buffer, and the tool can use auxiliary delimiter syntax elements to detect picture boundaries. To further reduce delay in low-latency decoding, the tool can reduce number of pictures in flight for multi-threaded decoding and reduce or remove jitter buffers. The tool receives encoded data, performs decoding according to the selected decoding mode to reconstruct pictures, and outputs the pictures for display.

摘要翻译： 用于减少视频解码中延迟延迟的技术和工具，用于强调低延迟的实时通信应用。 例如，诸如视频解码器的工具选择低延迟解码模式。 基于选择的解码模式，该工具调整输出定时确定，图像边界检测，飞行中的图像数和/或抖动缓冲器利用率。 对于低延迟解码，该工具可以使用帧计数语法元素为解码图像缓冲区设置初始输出延迟，并且该工具可以使用辅助分隔符语法元素来检测图像边界。 为了进一步减少低延迟解码的延迟，该工具可以减少飞行中用于多线程解码的图片数量，并减少或删除抖动缓冲区。 该工具接收编码数据，根据选择的解码模式执行解码以重构图像，并输出用于显示的图像。

公开(公告)号：US20110013889A1

公开(公告)日：2011-01-20

申请号：US12505444

申请日：2009-07-17

申请人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani ,  Naveen Thumpudi ,  Eric S. Christoffersen

发明人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani ,  Naveen Thumpudi ,  Eric S. Christoffersen

IPC分类号： H04N5/91 ,  H04N11/04 ,  H04N7/26

CPC分类号： H04N5/93 ,  H04N5/783 ,  H04N19/132 ,  H04N19/159 ,  H04N19/172 ,  H04N19/44 ,  H04N19/61 ,  H04N19/70 ,  H04N19/89 ,  H04N21/23424 ,  H04N21/4384 ,  H04N21/44016 ,  H04N21/485

摘要： A video bit stream with pictures comprising inter-coded content can be decoded upon receiving a channel start or file seek instruction. Pictures for beginning decoding and display of the bit stream can be selected based at least in part on one or more tuning parameters that set a preference between a latency of beginning to display video and possible defects in the displayed video. In some embodiments, to implement decoding upon a channel start or file seek, one or more types of data are generated for one or more pictures. For example, picture order counts are generated for pictures after a channel start or file seek operation. As another example, a decoder generates a frame number value that triggers re-initialization of a reference picture buffer before decoding after a channel start or file seek operation.

摘要翻译： 可以在接收到频道开始或文件搜索指令时解码包括帧间编码内容的图像的视频比特流。 可以至少部分地基于在开始显示视频的等待时间和显示的视频中的可能缺陷之间设置偏好的一个或多个调谐参数来选择用于开始解码和显示比特流的图片。 在一些实施例中，为了在信道开始或文件寻找中实现解码，为一个或多个图像生成一个或多个类型的数据。 例如，在通道开始或文件搜索操作之后，为图像生成图像顺序计数。 作为另一示例，解码器生成在通道开始或文件搜索操作之前解码之前触发参考图像缓冲器的重新初始化的帧编号值。

公开(公告)号：US20120154526A1

公开(公告)日：2012-06-21

申请号：US12972292

申请日：2010-12-17

申请人： Gang Ji ,  Shijun Sun ,  Yongjun Wu ,  Shyam Sadhwani

发明人： Gang Ji ,  Shijun Sun ,  Yongjun Wu ,  Shyam Sadhwani

IPC分类号： H04N13/04 ,  H04N13/00

CPC分类号： H04N13/334 ,  H04N13/106 ,  H04N13/15 ,  H04N13/341

摘要： Methods are disclosed for supporting stereo 3D video in computing devices. A computing device can receive stereo 3D video data employing a YUV color space and chroma subsampling, and can generate anaglyph video data therefrom. The anaglyph video data can be generated by unpacking the stereo 3D video data to left and right views and combining the left and right views into a single view via matrix transformation. The combining uses transform matrices that correspond to a video pipeline configuration. The transform matrix coefficients can depend on characteristics of the video pipeline components. Modified transform matrix coefficients can be used in response to changes in the video pipeline configuration. Video encoded in stereo 3D video data can be selected to be displayed in stereo 3D, anaglyph or monoscopic form, depending on user input and/or characteristics of video pipeline components.

摘要翻译： 公开了用于在计算设备中支持立体3D视频的方法。 计算设备可以使用YUV颜色空间和色度子采样来接收立体3D视频数据，并且可以从其生成浮雕视频数据。 可以通过将立体3D视频数据解包为左视图和右视图并通过矩阵变换将左视图和右视图组合成单个视图来生成浮雕视频数据。 组合使用对应于视频流水线配置的变换矩阵。 变换矩阵系数可以取决于视频管线组件的特性。 修改后的变换矩阵系数可用于响应视频流水线配置的变化。 可以根据用户输入和/或视频流水线部件的特性，选择以立体3D，浮雕形式或单视角形式显示立体3D视频数据编码的视频。

公开(公告)号：US20100128778A1

公开(公告)日：2010-05-27

申请号：US12323055

申请日：2008-11-25

申请人： Gang Ji ,  Naveen Thumpudi ,  Shyam Sadhwani ,  Yongjun Wu

发明人： Gang Ji ,  Naveen Thumpudi ,  Shyam Sadhwani ,  Yongjun Wu

IPC分类号： H04N7/30 ,  H04N7/26

CPC分类号： H04N19/42 ,  H04N19/44 ,  H04N19/61 ,  H04N19/89 ,  H04N19/93

摘要： Adjustment of hardware acceleration level in a video decoder utilizing hardware acceleration is described. Errors are detected in a bitstream as it is decoded using different levels of error detection based on decoding characteristics. A statistical analysis is performed on the error values as they are detected. In one technique, if the bitstream is categorized as fitting a high error rate state in a bitstream model, then hardware acceleration is dropped. In another technique, error statistics based on run-lengths of good and bad bitstream units are kept, and compared to predetermined thresholds. If the thresholds are exceeded, the hardware acceleration level is dropped. The level is dropped in order to take advantage of superior error handing abilities of software-based decoding over hardware-accelerated decoding.

摘要翻译： 描述了利用硬件加速度调整视频解码器中的硬件加速度水平。 在比特流中检测到错误，因为它使用基于解码特性的不同级别的错误检测被解码。 在检测到误差值时进行统计分析。 在一种技术中，如果比特流被分类为在比特列模型中拟合高错误率状态，则硬件加速度下降。 在另一种技术中，保持基于良好和不良比特流单元的游程长度的误差统计，并与预定阈值进行比较。 如果超过阈值，则硬件加速等级会下降。 为了利用基于硬件加速解码的基于软件的解码的优异的错误处理能力，该级别被丢弃。

公开(公告)号：US09131241B2

公开(公告)日：2015-09-08

申请号：US12323055

申请日：2008-11-25

申请人： Gang Ji ,  Naveen Thumpudi ,  Shyam Sadhwani ,  Yongjun Wu

发明人： Gang Ji ,  Naveen Thumpudi ,  Shyam Sadhwani ,  Yongjun Wu

IPC分类号： H04N5/85 ,  H04N19/42 ,  H04N19/61 ,  H04N19/44 ,  H04N19/89 ,  H04N19/93

CPC分类号： H04N19/42 ,  H04N19/44 ,  H04N19/61 ,  H04N19/89 ,  H04N19/93

摘要： Adjustment of hardware acceleration level in a video decoder utilizing hardware acceleration is described. Errors are detected in a bitstream as it is decoded using different levels of error detection based on decoding characteristics. A statistical analysis is performed on the error values as they are detected. In one technique, if the bitstream is categorized as fitting a high error rate state in a bitstream model, then hardware acceleration is dropped. In another technique, error statistics based on run-lengths of good and bad bitstream units are kept, and compared to predetermined thresholds. If the thresholds are exceeded, the hardware acceleration level is dropped. The level is dropped in order to take advantage of superior error handing abilities of software-based decoding over hardware-accelerated decoding.

摘要翻译： 描述了利用硬件加速度调整视频解码器中的硬件加速度水平。 在比特流中检测到错误，因为它使用基于解码特性的不同级别的错误检测被解码。 在检测到错误值时进行统计分析。 在一种技术中，如果比特流被分类为在比特列模型中拟合高错误率状态，则硬件加速度下降。 在另一种技术中，保持基于良好和不良比特流单元的游程长度的误差统计，并与预定阈值进行比较。 如果超过阈值，则硬件加速等级会下降。 为了利用基于硬件加速解码的基于软件的解码的优异的错误处理能力，该级别被丢弃。

公开(公告)号：US08860785B2

公开(公告)日：2014-10-14

申请号：US12972292

申请日：2010-12-17

申请人： Gang Ji ,  Shijun Sun ,  Yongjun Wu ,  Shyam Sadhwani

发明人： Gang Ji ,  Shijun Sun ,  Yongjun Wu ,  Shyam Sadhwani

IPC分类号： H04N13/04 ,  H04N13/00

CPC分类号： H04N13/334 ,  H04N13/106 ,  H04N13/15 ,  H04N13/341

摘要： Methods are disclosed for supporting stereo 3D video in computing devices. A computing device can receive stereo 3D video data employing a YUV color space and chroma subsampling, and can generate anaglyph video data therefrom. The anaglyph video data can be generated by unpacking the stereo 3D video data to left and right views and combining the left and right views into a single view via matrix transformation. The combining uses transform matrices that correspond to a video pipeline configuration. The transform matrix coefficients can depend on characteristics of the video pipeline components. Modified transform matrix coefficients can be used in response to changes in the video pipeline configuration. Video encoded in stereo 3D video data can be selected to be displayed in stereo 3D, anaglyph or monoscopic form, depending on user input and/or characteristics of video pipeline components.

摘要翻译： 公开了用于在计算设备中支持立体3D视频的方法。 计算设备可以使用YUV颜色空间和色度子采样来接收立体3D视频数据，并且可以从其生成浮雕视频数据。 可以通过将立体3D视频数据解包为左视图和右视图并通过矩阵变换将左视图和右视图组合成单个视图来生成浮雕视频数据。 组合使用对应于视频流水线配置的变换矩阵。 变换矩阵系数可以取决于视频管线组件的特性。 修改后的变换矩阵系数可用于响应视频流水线配置的变化。 可以根据用户输入和/或视频流水线部件的特性，选择以立体3D，浮雕形式或单视角形式显示立体3D视频数据编码的视频。

公开(公告)号：US08340510B2

公开(公告)日：2012-12-25

申请号：US12505444

申请日：2009-07-17

申请人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani ,  Naveen Thumpudi ,  Eric S. Christoffersen

发明人： Yongjun Wu ,  Gang Ji ,  Shyam Sadhwani ,  Naveen Thumpudi ,  Eric S. Christoffersen

IPC分类号： H04N5/93 ,  H04N5/783

CPC分类号： H04N5/93 ,  H04N5/783 ,  H04N19/132 ,  H04N19/159 ,  H04N19/172 ,  H04N19/44 ,  H04N19/61 ,  H04N19/70 ,  H04N19/89 ,  H04N21/23424 ,  H04N21/4384 ,  H04N21/44016 ,  H04N21/485

摘要： A video bit stream with pictures comprising inter-coded content can be decoded upon receiving a channel start or file seek instruction. Pictures for beginning decoding and display of the bit stream can be selected based at least in part on one or more tuning parameters that set a preference between a latency of beginning to display video and possible defects in the displayed video. In some embodiments, to implement decoding upon a channel start or file seek, one or more types of data are generated for one or more pictures. For example, picture order counts are generated for pictures after a channel start or file seek operation. As another example, a decoder generates a frame number value that triggers re-initialization of a reference picture buffer before decoding after a channel start or file seek operation.

摘要翻译： 可以在接收到频道开始或文件寻道指令时解码包含帧间编码内容的图像的视频比特流。 可以至少部分地基于在开始显示视频的等待时间和显示的视频中的可能缺陷之间设置偏好的一个或多个调谐参数来选择用于开始解码和显示比特流的图片。 在一些实施例中，为了在信道开始或文件寻找中实现解码，为一个或多个图像生成一个或多个类型的数据。 例如，在通道开始或文件搜索操作之后，为图像生成图像顺序计数。 作为另一示例，解码器生成在通道开始或文件搜索操作之前解码之前触发参考图像缓冲器的重新初始化的帧编号值。

公开(公告)号：US20120250772A1

公开(公告)日：2012-10-04

申请号：US13078852

申请日：2011-04-01

申请人： Wenfeng Gao ,  Yongjun Wu ,  Shyam Sadhwani

发明人： Wenfeng Gao ,  Yongjun Wu ,  Shyam Sadhwani

IPC分类号： H04N7/26

CPC分类号： H04N19/865 ,  H04N19/117 ,  H04N19/127 ,  H04N19/157 ,  H04N19/159 ,  H04N19/16 ,  H04N19/172 ,  H04N19/436 ,  H04N19/82 ,  H04N19/86

摘要： Multi-threaded implementations of deblock filtering improve encoding and/or decoding efficiency. For example, a video encoder or decoder partitions a video picture into multiple segments. The encoder/decoder selects between multiple different patterns for splitting operations of deblock filtering into multiple passes. The encoder/decoder organizes the deblock filtering as multiple tasks, where a given task includes the operations of one of the passes for one of the segments. The encoder/decoder then performs the tasks with multiple threads. The performance of the tasks is constrained by task dependencies which, in general, are based at least in part on which lines of the picture are in the respective segments and which deblock filtering operations are in the respective passes. The task dependencies can include a cross-pass, cross-segment dependency between a given pass of a given segment and an adjacent pass of an adjacent segment.

摘要翻译： 去块滤波的多线程实现提高了编码和/或解码效率。 例如，视频编码器或解码器将视频图像分割成多个段。 编码器/解码器在多个不同的模式之间进行选择，用于将去块滤波的分割操作分成多个通道。 编码器/解码器将去块滤波器组织为多个任务，其中给定任务包括其中一个段的一个通路的操作。 然后，编码器/解码器使用多个线程执行任务。 任务的执行受到任务依赖性的约束，通常，至少部分地基于图片的哪些行在相应的段中，以及哪些去块过滤操作在相应的遍历中。 任务依赖性可以包括给定段的给定遍和相邻段的相邻段之间的交叉遍，跨段依赖性。

公开(公告)号：US20130051478A1

公开(公告)日：2013-02-28

申请号：US13223079

申请日：2011-08-31

申请人： Yongjun Wu ,  Shyam Sadhwani

发明人： Yongjun Wu ,  Shyam Sadhwani

IPC分类号： H04N7/26

CPC分类号： H04N19/423 ,  H04N19/0049 ,  H04N19/127 ,  H04N19/136 ,  H04N19/159 ,  H04N19/172 ,  H04N19/176 ,  H04N19/426 ,  H04N19/44

摘要： Techniques and tools described herein help manage memory efficiently during video decoding, especially when multiple video clips are concurrently decoded. For example, with clip-adaptive memory usage, a decoder determines first memory usage settings expected to be sufficient for decoding of a video clip. The decoder also determines second memory usage settings known to be sufficient for decoding of the clip. During decoding, memory usage is initially set according to the first settings. Memory usage is adaptively increased during decoding, subject to theoretical limits in the second settings. With adaptive early release of side information, the decoder can release side information memory for a picture earlier than the decoder releases image plane memory for the picture. The decoder can also adapt memory usage for decoded transform coefficients depending on whether the coefficients are for intra-coded blocks or inter-coded blocks, and also exploit the relative sparseness of non-zero coefficient values.

摘要翻译： 这里描述的技术和工具有助于在视频解码期间有效地管理存储器，特别是当多个视频片段被同时解码时。 例如，利用剪辑自适应存储器使用，解码器确定预期对视频剪辑的解码足够的第一存储器使用设置。 解码器还确定已知足以解码剪辑的第二存储器使用设置。 在解码期间，最初根据第一个设置设置内存使用。 存储器使用在解码期间自适应地增加，受到第二设置中的理论限制。 利用辅助信息的适应性早期释放，解码器可以在解码器释放用于图像的图像平面存储器之前释放用于图像的侧信息存储器。 解码器还可以根据系数是用于帧内编码块还是帧间编码块来适应解码变换系数的存储器使用，并且还利用非零系数值的相对稀疏度。