公开(公告)号：US20200186796A1

公开(公告)日：2020-06-11

申请号：US16295176

申请日：2019-03-07

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: H04N19/107 ,  H04N19/105 ,  G06N20/00 ,  H04N19/184 ,  H04N19/154

Abstract: Video coding may include generating, by a processor, a decoded frame by decoding a current frame from an encoded bitstream and outputting a reconstructed frame based on the decoded frame. Decoding includes identifying a current encoded block from the current frame, identifying a prediction coding model for the current block, wherein the prediction coding model is a machine learning prediction coding model from a plurality of machine learning prediction coding models, identifying reference values for decoding the current block based on the prediction coding model, obtaining prediction values based on the prediction coding model and the reference values, generating a decoded block corresponding to the current encoded block based on the prediction values, and including the decoded block in the decoded frame.

公开(公告)号：US20200184603A1

公开(公告)日：2020-06-11

申请号：US16515226

申请日：2019-07-18

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: G06T5/00 ,  G06T5/50 ,  G06N3/08 ,  G06N3/04 ,  G06N20/20 ,  G06T9/00 ,  G06T3/40 ,  H04N19/70 ,  H04N19/176 ,  G06K9/62

Abstract: Guided restoration is used to restore video data degraded from a video frame. The video frame is divided into restoration units (RUs) which each correspond to one or more blocks of the video frame. Restoration schemes are selected for each RU. The restoration schemes may indicate to use one of a plurality of neural networks trained for the guided restoration. Alternatively, the restoration schemes may indicate to use a neural network and a filter-based restoration tool. The video frame is then restored by processing each RU according to the respective selected restoration scheme. During encoding, the restored video frame is encoded to an output bitstream, and the use of the selected restoration schemes may be signaled within the output bitstream. During decoding, the restored video frame is output to an output video stream.

公开(公告)号：US20190394482A1

公开(公告)日：2019-12-26

申请号：US16018105

申请日：2018-06-26

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Andrew Simpson

IPC: H04N19/50 ,  H04N7/01 ,  H04N19/176 ,  H04N19/184 ,  H04N19/61 ,  H04N19/117

Abstract: Systems and methods are disclosed for encoding and decoding video. For example, methods may include accessing an encoded bitstream; decoding loop restoration parameters in the encoded bitstream; after reconstruction of an image at a second resolution based on data of the encoded bitstream, upscaling the reconstructed image to obtain an upscaled reconstructed image at a first resolution, wherein the second resolution is less than the first resolution in at least one dimension; and applying loop restoration filtering to the upscaled reconstructed image using the loop restoration parameters to obtain a loop restored image at the first resolution.

公开(公告)号：US20240098280A1

公开(公告)日：2024-03-21

申请号：US18272862

申请日：2021-01-19

Applicant: Google LLC

Inventor： Urvang Joshi ,  Yue Chen ,  Sarah Parker ,  Elliott Karpilovsky ,  Debargha Mukherjee

IPC: H04N19/176 ,  H04N19/30

CPC classification number: H04N19/176 ,  H04N19/30

Abstract: Image coding using guided machine learning restoration may include obtaining reconstructed frame data by decoding, obtaining a restored frame by restoring the reconstructed frame, and outputting the restored frame. Obtaining the restored frame may include obtaining a reconstructed block, obtaining guide parameter values, obtaining a restored block, and including the restored block in the restored frame. Obtaining the restored block may include inputting the reconstructed block to an input layer of a trained guided convolutional neural network, wherein the neural network is constrained such that an output layer has a defined cardinality of channels, obtaining, from the output layer, neural network output channel predictions, obtaining a guided neural network prediction as a linear combination of the guide parameter values and the neural network output channel predictions, and generating the restored block using the guided neural network prediction.

公开(公告)号：US20230179789A1

公开(公告)日：2023-06-08

申请号：US18155224

申请日：2023-01-17

Applicant: Google LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Andrew Simpson

IPC: H04N19/50 ,  H04N7/01 ,  H04N19/117 ,  H04N19/184 ,  H04N19/61 ,  H04N19/176

CPC classification number: H04N19/50 ,  H04N7/0117 ,  H04N19/117 ,  H04N19/184 ,  H04N19/619 ,  H04N19/176

Abstract: A super-resolution coding mode is described. An encoded image can be decoded from an encoded bitstream stored on a non-transitory computer-readable storage medium. A flag can indicate whether an image was encoded using the super-resolution mode at a first resolution. Responsive to the flag indicating that the image was encoded using the super-resolution mode, bits indicating an amount of scaling of the image are included. The image is decoded from the encoded bitstream to obtain a reconstructed image at the first resolution, and the reconstructed image is upscaled to a second resolution using the amount of scaling to obtain an upscaled reconstructed image. The second resolution is higher than the first resolution. Loop restoration parameters within the bitstream can used for look restoration filtering of the upscaled reconstructed image to obtain a loop restored image at the second resolution.

公开(公告)号：US20210409705A1

公开(公告)日：2021-12-30

申请号：US16912767

申请日：2020-06-26

Applicant: GOOGLE LLC

Inventor： Sarah Parker ,  Debargha Mukherjee ,  Yue Chen ,  Elliott Karpilovsky ,  Urvang Joshi

IPC: H04N19/119 ,  H04N19/12 ,  H04N19/61 ,  H04N19/176 ,  G06N20/00

Abstract: Transform-level partitioning of a prediction residual block is performed to improve compression efficiency of video data. During encoding, a prediction residual block is generated responsive to prediction-level partitioning performed against a video block, a transform block partition type to use is determined based on the prediction residual block, a non-recursive transform-level partitioning is performed against the prediction residual block according to the transform block partition type, and transform blocks generated as a result of the transform-level partitioning are encoded to a bitstream. During decoding, a symbol representative of the transform block partition type used to encode transform blocks is derived from the bitstream, inverse transformed blocks are produced by inverse transforming encoded video data associated with the prediction residual block, and the prediction residual block is reproduced according to the transform block partition type and used to reconstruct the video block, which is output within an output video stream.

公开(公告)号：US11039131B2

公开(公告)日：2021-06-15

申请号：US16831943

申请日：2020-03-27

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/11 ,  H04N19/593 ,  H04N19/176 ,  H04N19/61 ,  H04N19/59

Abstract: An apparatus for coding a block of a frame using intra-prediction includes a memory and a processor. The processor is configured to execute instructions stored in the memory to obtain an intra-prediction mode for coding the block of the frame; select a transform type for coding a transform block of a residual block, which results from predicting the block using the intra-prediction mode; and code the transform block using the transform type. To select the transform type includes to, in a case where the intra-prediction mode is a SMOOTH_PRED, select a ADST_ADST transform type; in a case where the intra-prediction mode is a SMOOTH_H_PRED, select a DCT_ADST transform type; and in a case where the intra-prediction mode is a SMOOTH_V_PRED, select a ADST_DCT transform type.

公开(公告)号：US20200228800A1

公开(公告)日：2020-07-16

申请号：US16831943

申请日：2020-03-27

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/11 ,  H04N19/593 ,  H04N19/176 ,  H04N19/61

Abstract: An apparatus for coding a block of a frame using intra-prediction includes a memory and a processor. The processor is configured to execute instructions stored in the memory to obtain an intra-prediction mode for coding the block of the frame; select a transform type for coding a transform block of a residual block, which results from predicting the block using the intra-prediction mode; and code the transform block using the transform type. To select the transform type includes to, in a case where the intra-prediction mode is a SMOOTH_PRED, select a ADST_ADST transform type; in a case where the intra-prediction mode is a SMOOTH_H_PRED, select a DCT_ADST transform type; and in a case where the intra-prediction mode is a SMOOTH_V_PRED, select a ADST_DCT transform type.

公开(公告)号：US20190379889A1

公开(公告)日：2019-12-12

申请号：US16004929

申请日：2018-06-11

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/122 ,  H04N19/176 ,  H04N19/124 ,  H04N19/18 ,  H04N19/159 ,  H04N19/82

Abstract: Improved transforms are used to encode and decode large video and image blocks. During encoding, a prediction residual block having a large size (e.g., larger than 32x32) is generated. The pixel values of the prediction residual block are transformed to produce transform coefficients. After determining that the transform coefficients exceed a threshold cardinality representative of a maximum transform block size (e.g., 32x32), a number of the transform coefficients are discarded such that a remaining number of transform coefficients does not exceed the threshold cardinality. A transform block is then generated using the remaining number. During decoding, after determining that the transform coefficients exceed the threshold cardinality, a number of new coefficients are added to the transform coefficients such that a total number of transform coefficients exceeds the threshold cardinality. The transform coefficients are then inverse transformed into a prediction residual block having a large size.

公开(公告)号：US20230291925A1

公开(公告)日：2023-09-14

申请号：US18008209

申请日：2020-07-01

Applicant: Google LLC

Inventor： Debargha Mukherjee ,  Yue Chen ,  Urvang Joshi ,  Sarah Parker ,  Elliott Karpilovsky ,  Hui Su

IPC: H04N19/52 ,  H04N19/176 ,  H04N19/593 ,  H04N19/105

CPC classification number: H04N19/52 ,  H04N19/176 ,  H04N19/593 ,  H04N19/105

Abstract: Video coding in accordance with an inter-intra prediction model may include coding an inter-prediction motion vector for a current block of a current frame, obtaining spatial block-context pixels oriented relative to the current block, generating an inter-prediction block, generating a corresponding set of reference block-context pixels oriented relative to the inter-prediction block, identifying inter-intra prediction parameters that correspond with minimizing error between the spatial block-context pixels and the reference block-context pixels, generating a prediction block for the current block by, for a current pixel of the current block, obtaining an inter-prediction pixel, determining a predictor for the current pixel using a combination of the inter-prediction pixel and the inter-intra prediction parameters, and including the predictor in the prediction block.