Methods and Apparatus for Learning Based Adaptive Real-time Streaming

    公开(公告)号:US20200162535A1

    公开(公告)日:2020-05-21

    申请号:US16687739

    申请日:2019-11-19

    IPC分类号: H04L29/06 H04L12/26 G06N3/08

    摘要: This invention discloses a deep reinforcement learning based adaptive bitrate selection method and system for real-time streaming, where deep reinforcement learning neural networks are utilized to receive states observations and make bitrate decisions. Simulation is constructed to provide network states including network QoS and playback status to agents and compute accumulated rewards according to the bitrate actions made by agents. ARS balances a variety of QoE goals to determine the accumulated rewards. ARS also enables multiple agents to be trained concurrently and conducts training process in a simulation environment to accelerate the training speed. In addition, ARS supports training ABR algorithm both online and offline.

    Method And Apparatus Of Collaborative Video Processing Through Learned Resolution Scaling

    公开(公告)号:US20200162789A1

    公开(公告)日:2020-05-21

    申请号:US16688786

    申请日:2019-11-19

    申请人: Zhan Ma Ming Lu

    发明人: Zhan Ma Ming Lu

    摘要: In a collaborative video processing method and system, a high resolution video input is optionally downscaled to a low resolution video using a down-sampling filter, followed by an end-to-end video coding system to encode the low resolution video for streaming over the Internet. The original high resolution is obtained at the client end by upscaling the low resolution video using a deep learning based high resolution scaling model, which can be trained in a pre-defined progressive order with low resolution videos having different compression parameters and downscaling factors.

    Method and Apparatus for the Single Input Multiple Output (SIMO) Media Adaptation
    4.
    发明申请
    Method and Apparatus for the Single Input Multiple Output (SIMO) Media Adaptation 审中-公开
    单输入多输出(SIMO)媒体适应的方法和装置

    公开(公告)号:US20170064311A1

    公开(公告)日:2017-03-02

    申请号:US15249281

    申请日:2016-08-26

    申请人: Zhan Ma

    发明人: Zhan Ma

    摘要: A method and apparatus for the single input multiple output based media adaptation is disclosed. In one embodiment, such adaption is performed in two steps. On step 1, content correlation between different compression schemes is used to perform the inter-format adaptation of a stream of a compression format to an intermediate output stream of another compression scheme with the same quality level. On step 2, content correlation between different quality levels is used to perform the intra-format adaptation of the intermediate output stream to multiple output streams at different quality levels with the same compression format. In one embodiment, content correlation is used to limit the search for mode candidates when performing both steps.

    摘要翻译: 公开了一种用于基于单输入多输出的媒体适配的方法和装置。 在一个实施例中,这种适应在两个步骤中执行。 在步骤1中,使用不同压缩方案之间的内容相关性来执行压缩格式的流的格式间格式适配到具有相同质量级别的另一压缩方案的中间输出流。 在步骤2中,使用不同质量级别之间的内容相关性,以相同的压缩格式,以不同的质量级别对中间输出流进行格式内适配。 在一个实施例中,当执行两个步骤时,使用内容相关来限制对模式候选的搜索。

    Methods and Apparatus for Motion Compensation with Smooth Reference Frame in Bit Depth Scalability
    7.
    发明申请
    Methods and Apparatus for Motion Compensation with Smooth Reference Frame in Bit Depth Scalability 有权
    用于运动补偿的平滑参考框架在位深度可扩展性的方法和装置

    公开(公告)号:US20110293013A1

    公开(公告)日:2011-12-01

    申请号:US13138342

    申请日:2009-12-11

    IPC分类号: H04N7/32 H04N7/26

    摘要: Methods and apparatus are provided for motion compensation with a smooth reference frame in bit depth scalability. An apparatus includes an encoder for encoding picture data for at least a portion of a picture by generating an inter-layer residue prediction for the portion using an inverse tone mapping operation performed in the pixel domain for bit depth scalability. The inverse tone mapping operation is shifted from a residue domain to the pixel domain.

    摘要翻译: 为比特深度可扩展性的平滑参考帧提供运动补偿的方法和装置。 一种装置包括:编码器,用于通过使用在像素域中进行的逆色调映射操作生成用于比特深度可缩放性的部分的该部分的层间残差预测,对图像的至少一部分进行图像数据的编码。 逆色调映射操作从残留域移动到像素域。

    Apparatus and method for robust low-complexity video fingerprinting
    9.
    发明授权
    Apparatus and method for robust low-complexity video fingerprinting 有权
    强大的低复杂度视频指纹识别装置和方法

    公开(公告)号:US08995708B2

    公开(公告)日:2015-03-31

    申请号:US13605718

    申请日:2012-09-06

    IPC分类号: G06K9/00 G06F17/30

    CPC分类号: G06F17/30784 G06K9/00751

    摘要: An apparatus and method for video fingerprinting are provided. The method includes, for each frame of a video sequence including a plurality of frames, removing a portion of the frame, dividing a remaining portion of the frame into blocks, dividing each block into sub-blocks, computing a block level feature as a mean of pixels in each sub-block within the block, concatenating all block level features in the frame, and concatenating features of all frames in the video sequence.

    摘要翻译: 提供了一种用于视频指纹识别的装置和方法。 该方法包括对于包括多个帧的视频序列的每个帧,去除帧的一部分,将帧的剩余部分划分成块,将每个块划分成子块,计算块级特征作为平均值 在块内的每个子块中的像素,连接帧中的所有块级特征,以及连接视频序列中所有帧的特征。

    TECHNIQUES FOR PERCEPTUAL ENCODING OF VIDEO FRAMES
    10.
    发明申请
    TECHNIQUES FOR PERCEPTUAL ENCODING OF VIDEO FRAMES 有权
    用于视频编码视频框架的技术

    公开(公告)号:US20110122942A1

    公开(公告)日:2011-05-26

    申请号:US12951035

    申请日:2010-11-20

    IPC分类号: H04N7/26

    摘要: In a video encoder, pixel values of a macro-block are processed to determine an activity measure indicative of the type of content in the macro-block. Several techniques are employed for determining the activity measure of a macro-block. In an embodiment, a default quantization scale for quantizing a macro-block is modified based on the activity measure of the macro-block. In another embodiment, the macro-block is classified into one of multiple classes based on its activity measure. The default quantization scale for quantizing the macro-block is modified based on the classification of the macro-block. In yet another embodiment, an encoding mode to be used for encoding a macro-block is also determined on the basis of the class of the macro-block. Several of the techniques exploit the fact that the human visual system (HVS) has different sensitivities in perceiving a (rendered) macro-block or video frame, depending on the type of macro-block content.

    摘要翻译: 在视频编码器中,处理宏块的像素值以确定指示宏块中的内容类型的活动度量。 采用几种技术来确定宏块的活动度量。 在一个实施例中,基于宏块的活动度量来修改用于量化宏块的默认量化尺度。 在另一个实施例中,基于其活动度量将宏块分类为多个类别之一。 基于宏块的分类修改用于量化宏块的默认量化尺度。 在另一个实施例中,还根据宏块的类确定用于编码宏块的编码模式。 几种技术利用人类视觉系统(HVS)根据宏块内容的类型在感知(渲染)宏块或视频帧时具有不同的灵敏度。