公开(公告)号：US20220345704A1

公开(公告)日：2022-10-27

申请号：US17860585

申请日：2022-07-08

Applicant: GOOGLE LLC

Inventor： Sarah Parker ,  Debargha Mukherjee ,  Yue Chen ,  Elliott Karpilovsky ,  Urvang Joshi

IPC: H04N19/119 ,  H04N19/12 ,  G06N20/00 ,  H04N19/176 ,  H04N19/61

Abstract: Transform-level partitioning of a prediction residual block is performed to improve compression efficiency of video data. During encoding, a prediction residual block is generated responsive to prediction-level partitioning performed against a video block, a transform block partition type to use is determined based on the prediction residual block, a non-recursive transform-level partitioning is performed against the prediction residual block according to the transform block partition type, and transform blocks generated as a result of the transform-level partitioning are encoded to a bitstream. During decoding, a symbol representative of the transform block partition type used to encode transform blocks is derived from the bitstream, inverse transformed blocks are produced by inverse transforming encoded video data associated with the prediction residual block, and the prediction residual block is reproduced according to the transform block partition type and used to reconstruct the video block, which is output within an output video stream.

公开(公告)号：US11405653B2

公开(公告)日：2022-08-02

申请号：US16666676

申请日：2019-10-29

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee

IPC: H04N19/86 ,  H04N19/182 ,  H04N19/46 ,  H04N19/80 ,  H04N19/174 ,  H04N19/85 ,  H04N19/192 ,  G06T5/00 ,  H04N19/117 ,  H04N19/136 ,  H04N19/176 ,  H04N19/463

Abstract: A method includes generating, using first restoration parameters, a first guide tile for a degraded tile of the degraded frame, the degraded tile corresponding to a source tile of the source frame; generating, using second restoration parameters, a second guide tile for the degraded tile of the degraded frame, the second restoration parameters being different from the first restoration parameters; determining a first tile difference between the source tile and the first guide tile; determining a second tile difference between the source tile and the second guide tile; calculating projection parameters that minimize a difference between a restored tile of the degraded tile and the source tile; and encoding, in an encoded bitstream, the projection parameters. The difference between the restored tile of the degraded tile and the source tile is a linear combination, using the projection parameters, of the first tile difference and the second tile difference.

公开(公告)号：US11388401B2

公开(公告)日：2022-07-12

申请号：US16912767

申请日：2020-06-26

Applicant: GOOGLE LLC

Inventor： Sarah Parker ,  Debargha Mukherjee ,  Yue Chen ,  Elliott Karpilovsky ,  Urvang Joshi

IPC: H04N19/119 ,  H04N19/12 ,  G06N20/00 ,  H04N19/176 ,  H04N19/61

Abstract: Transform-level partitioning of a prediction residual block is performed to improve compression efficiency of video data. During encoding, a prediction residual block is generated responsive to prediction-level partitioning performed against a video block, a transform block partition type to use is determined based on the prediction residual block, a non-recursive transform-level partitioning is performed against the prediction residual block according to the transform block partition type, and transform blocks generated as a result of the transform-level partitioning are encoded to a bitstream. During decoding, a symbol representative of the transform block partition type used to encode transform blocks is derived from the bitstream, inverse transformed blocks are produced by inverse transforming encoded video data associated with the prediction residual block, and the prediction residual block is reproduced according to the transform block partition type and used to reconstruct the video block, which is output within an output video stream.

公开(公告)号：US20220217336A1

公开(公告)日：2022-07-07

申请号：US17700298

申请日：2022-03-21

Applicant: Google LLC

Inventor： Lester Lu ,  Debargha Mukherjee ,  Elliott Karpilovsky

IPC: H04N19/107 ,  H04N19/122 ,  H04N19/176 ,  H04N19/50

Abstract: Decoding video data includes, for a block encoded using a prediction mode, determining a transform mode for the block using the prediction mode. The transform mode is a first mode when the prediction mode is an inter-prediction mode and is a second mode when the prediction mode is an intra-prediction mode. The first mode is an available first transform type that is a combination of transforms selected from first fixed transforms and first learned transforms that each comprise a respective transformation matrix generated iteratively using blocks predicted using the inter-prediction mode. The second mode is an available second transform type that is a combination of transforms selected from second fixed transforms, which is a proper subset of the first fixed transforms, and a second learned transform comprising a transformation matrix that is generated iteratively using blocks predicted using the intra-prediction mode. Decoding the block uses the prediction and transform modes.

公开(公告)号：US11343528B2

公开(公告)日：2022-05-24

申请号：US17073892

申请日：2020-10-19

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  James Bankoski ,  Yue Chen ,  Yuxin Liu ,  Sarah Parker

IPC: H04N11/02 ,  H04N19/517 ,  H04N19/46 ,  H04N19/136 ,  H04N19/105 ,  H04N19/139 ,  H04N19/176 ,  H04N19/182

Abstract: Generating a compound predictor block of a current block of video can include generating, for the current block, predictor blocks comprising a first predictor block including first predictor pixels and a second predictor block including second predictor pixels; using at least a subset of the first predictor pixels to determine a first weight for a first predictor pixel of the first predictor pixels; obtaining a second weight for a second predictor pixel of the second predictor pixels, where the second predictor pixel is co-located with the first predictor pixel; and generating the compound predictor block by combining the first predictor block and the second predictor block, where the predictor block includes a weighted pixel that is determined using a weighted sum of the first predictor pixel and the second predictor pixel using the first weight and the second weight, respectively.

公开(公告)号：US20210329245A1

公开(公告)日：2021-10-21

申请号：US17359767

申请日：2021-06-28

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/122 ,  H04N19/176 ,  H04N19/82 ,  H04N19/18 ,  H04N19/159 ,  H04N19/124

Abstract: Improved transforms are used to encode and decode large video and image blocks. During encoding, a prediction residual block having a large size (e.g., larger than 32×32) is generated. The pixel values of the prediction residual block are transformed to produce transform coefficients. After determining that the transform coefficients exceed a threshold cardinality representative of a maximum transform block size (e.g., 32×32), a number of the transform coefficients are discarded such that a remaining number of transform coefficients does not exceed the threshold cardinality. A transform block is then generated using the remaining number. During decoding, after determining that the transform coefficients exceed the threshold cardinality, a number of new coefficients are added to the transform coefficients such that a total number of transform coefficients exceeds the threshold cardinality. The transform coefficients are then inverse transformed into a prediction residual block having a large size.

公开(公告)号：US11051018B2

公开(公告)日：2021-06-29

申请号：US17012093

申请日：2020-09-04

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/122 ,  H04N19/176 ,  H04N19/82 ,  H04N19/18 ,  H04N19/159 ,  H04N19/124

Abstract: Improved transforms are used to encode and decode large video and image blocks. During encoding, a prediction residual block having a large size (e.g., larger than 32×32) is generated. The pixel values of the prediction residual block are transformed to produce transform coefficients. After determining that the transform coefficients exceed a threshold cardinality representative of a maximum transform block size (e.g., 32×32), a number of the transform coefficients are discarded such that a remaining number of transform coefficients does not exceed the threshold cardinality. A transform block is then generated using the remaining number. During decoding, after determining that the transform coefficients exceed the threshold cardinality, a number of new coefficients are added to the transform coefficients such that a total number of transform coefficients exceeds the threshold cardinality. The transform coefficients are then inverse transformed into a prediction residual block having a large size.

公开(公告)号：US20200260112A1

公开(公告)日：2020-08-13

申请号：US16861299

申请日：2020-04-29

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Emil Keyder ,  Michele Covell ,  Chen Wang ,  Sarah Parker ,  Ramin Zabih

IPC: H04N19/527 ,  H04N19/147 ,  H04N19/573 ,  H04N19/543 ,  H04N19/167 ,  H04N19/17 ,  H04N19/137 ,  H04N19/109 ,  H04N19/192 ,  H04N19/119 ,  H04N19/176 ,  H04N19/124 ,  H04N19/159 ,  H04N19/44 ,  H04N19/172 ,  G06T7/246

Abstract: An apparatus for encoding a current frame of a video. The apparatus includes a memory and a processor. The processor is configured to execute instructions stored in the memory to generate, for each reference frame of a subset of available reference frames, at least one respective candidate global motion model (GMM); partition the current frame into blocks; generate an aggregated residual frame for the current frame; and encode the respective residual blocks in a compressed bitstream. To generate the aggregated residual frame includes to select, for predicting each block of the blocks, a respective selected GMM, where the respective selected GMM corresponds to the one of the at least one respective candidate GMMs that minimizes a total error associated with the aggregated residual frame; and obtain respective residual blocks for the block.

公开(公告)号：US20200228820A1

公开(公告)日：2020-07-16

申请号：US16836078

申请日：2020-03-31

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Andrew Simpson

IPC: H04N19/50 ,  H04N7/01 ,  H04N19/117 ,  H04N19/184 ,  H04N19/61 ,  H04N19/176

Abstract: A super-resolution coding mode is described. Encoded image can be decoded by decoding, from an encoded bitstream, a flag indicating whether an image was encoded using the super-resolution mode. The image is encoded at a first resolution. Responsive to the flag indicating that the image was encoded using the super-resolution mode, bits indicating an amount of scaling of the image are decoded. The image is decoded from the encoded bitstream to obtain a reconstructed image at the first resolution, and the reconstructed image is upscaled to a second resolution using the amount of scaling to obtain an upscaled reconstructed image. The second resolution is higher than the first resolution. Loop restoration filtering is applied to the upscaled reconstructed image using loop restoration parameters to obtain a loop restored image at the second resolution.

公开(公告)号：US20200186808A1

公开(公告)日：2020-06-11

申请号：US16266662

申请日：2019-02-04

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Hui Su

IPC: H04N19/147 ,  H04N19/124 ,  H04N19/159 ,  H04N19/96 ,  H04N19/176 ,  H04L29/06

Abstract: A method for encoding a block of a video stream includes generating, using pixel values of the block, block features for the block; for each candidate encoding mode of candidate encoding modes, generating, using the block features and the each candidate encoding mode as inputs to a machine-learning module, a respective encoding cost; selecting, based on the respective encoding costs, a predetermined number of the candidate encoding modes; selecting, based on the respective encoding costs of the at least some encoding modes, a best mode for encoding the block; and encoding, in a compressed bitstream, the block using the best mode.