公开(公告)号：US11689726B2

公开(公告)日：2023-06-27

申请号：US16516784

申请日：2019-07-19

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: H04N19/147 ,  H04N19/184 ,  H04N19/59 ,  G06N3/04

CPC classification number: H04N19/147 ,  G06N3/04 ,  H04N19/184 ,  H04N19/59

Abstract: A hybrid apparatus for coding a video stream includes a first encoder. The first encoder includes a neural network having at least one hidden layer, and the neural network receives source data from the video stream at a first hidden layer of the at least one hidden layer, receives side information correlated with the source data at the first hidden layer, and generates guided information using the source data and the side information. The first encoder outputs the guided information and the side information for a decoder to reconstruct the source data.

公开(公告)号：US20220345704A1

公开(公告)日：2022-10-27

申请号：US17860585

申请日：2022-07-08

Applicant: GOOGLE LLC

Inventor： Sarah Parker ,  Debargha Mukherjee ,  Yue Chen ,  Elliott Karpilovsky ,  Urvang Joshi

IPC: H04N19/119 ,  H04N19/12 ,  G06N20/00 ,  H04N19/176 ,  H04N19/61

Abstract: Transform-level partitioning of a prediction residual block is performed to improve compression efficiency of video data. During encoding, a prediction residual block is generated responsive to prediction-level partitioning performed against a video block, a transform block partition type to use is determined based on the prediction residual block, a non-recursive transform-level partitioning is performed against the prediction residual block according to the transform block partition type, and transform blocks generated as a result of the transform-level partitioning are encoded to a bitstream. During decoding, a symbol representative of the transform block partition type used to encode transform blocks is derived from the bitstream, inverse transformed blocks are produced by inverse transforming encoded video data associated with the prediction residual block, and the prediction residual block is reproduced according to the transform block partition type and used to reconstruct the video block, which is output within an output video stream.

公开(公告)号：US11388401B2

公开(公告)日：2022-07-12

申请号：US16912767

申请日：2020-06-26

Applicant: GOOGLE LLC

Inventor： Sarah Parker ,  Debargha Mukherjee ,  Yue Chen ,  Elliott Karpilovsky ,  Urvang Joshi

IPC: H04N19/119 ,  H04N19/12 ,  G06N20/00 ,  H04N19/176 ,  H04N19/61

Abstract: Transform-level partitioning of a prediction residual block is performed to improve compression efficiency of video data. During encoding, a prediction residual block is generated responsive to prediction-level partitioning performed against a video block, a transform block partition type to use is determined based on the prediction residual block, a non-recursive transform-level partitioning is performed against the prediction residual block according to the transform block partition type, and transform blocks generated as a result of the transform-level partitioning are encoded to a bitstream. During decoding, a symbol representative of the transform block partition type used to encode transform blocks is derived from the bitstream, inverse transformed blocks are produced by inverse transforming encoded video data associated with the prediction residual block, and the prediction residual block is reproduced according to the transform block partition type and used to reconstruct the video block, which is output within an output video stream.

公开(公告)号：US20210329245A1

公开(公告)日：2021-10-21

申请号：US17359767

申请日：2021-06-28

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/122 ,  H04N19/176 ,  H04N19/82 ,  H04N19/18 ,  H04N19/159 ,  H04N19/124

Abstract: Improved transforms are used to encode and decode large video and image blocks. During encoding, a prediction residual block having a large size (e.g., larger than 32×32) is generated. The pixel values of the prediction residual block are transformed to produce transform coefficients. After determining that the transform coefficients exceed a threshold cardinality representative of a maximum transform block size (e.g., 32×32), a number of the transform coefficients are discarded such that a remaining number of transform coefficients does not exceed the threshold cardinality. A transform block is then generated using the remaining number. During decoding, after determining that the transform coefficients exceed the threshold cardinality, a number of new coefficients are added to the transform coefficients such that a total number of transform coefficients exceeds the threshold cardinality. The transform coefficients are then inverse transformed into a prediction residual block having a large size.

公开(公告)号：US11051018B2

公开(公告)日：2021-06-29

申请号：US17012093

申请日：2020-09-04

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/122 ,  H04N19/176 ,  H04N19/82 ,  H04N19/18 ,  H04N19/159 ,  H04N19/124

Abstract: Improved transforms are used to encode and decode large video and image blocks. During encoding, a prediction residual block having a large size (e.g., larger than 32×32) is generated. The pixel values of the prediction residual block are transformed to produce transform coefficients. After determining that the transform coefficients exceed a threshold cardinality representative of a maximum transform block size (e.g., 32×32), a number of the transform coefficients are discarded such that a remaining number of transform coefficients does not exceed the threshold cardinality. A transform block is then generated using the remaining number. During decoding, after determining that the transform coefficients exceed the threshold cardinality, a number of new coefficients are added to the transform coefficients such that a total number of transform coefficients exceeds the threshold cardinality. The transform coefficients are then inverse transformed into a prediction residual block having a large size.

公开(公告)号：US20200228820A1

公开(公告)日：2020-07-16

申请号：US16836078

申请日：2020-03-31

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Andrew Simpson

IPC: H04N19/50 ,  H04N7/01 ,  H04N19/117 ,  H04N19/184 ,  H04N19/61 ,  H04N19/176

Abstract: A super-resolution coding mode is described. Encoded image can be decoded by decoding, from an encoded bitstream, a flag indicating whether an image was encoded using the super-resolution mode. The image is encoded at a first resolution. Responsive to the flag indicating that the image was encoded using the super-resolution mode, bits indicating an amount of scaling of the image are decoded. The image is decoded from the encoded bitstream to obtain a reconstructed image at the first resolution, and the reconstructed image is upscaled to a second resolution using the amount of scaling to obtain an upscaled reconstructed image. The second resolution is higher than the first resolution. Loop restoration filtering is applied to the upscaled reconstructed image using loop restoration parameters to obtain a loop restored image at the second resolution.

公开(公告)号：US20200186808A1

公开(公告)日：2020-06-11

申请号：US16266662

申请日：2019-02-04

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Hui Su

IPC: H04N19/147 ,  H04N19/124 ,  H04N19/159 ,  H04N19/96 ,  H04N19/176 ,  H04L29/06

Abstract: A method for encoding a block of a video stream includes generating, using pixel values of the block, block features for the block; for each candidate encoding mode of candidate encoding modes, generating, using the block features and the each candidate encoding mode as inputs to a machine-learning module, a respective encoding cost; selecting, based on the respective encoding costs, a predetermined number of the candidate encoding modes; selecting, based on the respective encoding costs of the at least some encoding modes, a best mode for encoding the block; and encoding, in a compressed bitstream, the block using the best mode.