公开(公告)号：US11689726B2

公开(公告)日：2023-06-27

申请号：US16516784

申请日：2019-07-19

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: H04N19/147 ,  H04N19/184 ,  H04N19/59 ,  G06N3/04

CPC classification number: H04N19/147 ,  G06N3/04 ,  H04N19/184 ,  H04N19/59

Abstract: A hybrid apparatus for coding a video stream includes a first encoder. The first encoder includes a neural network having at least one hidden layer, and the neural network receives source data from the video stream at a first hidden layer of the at least one hidden layer, receives side information correlated with the source data at the first hidden layer, and generates guided information using the source data and the side information. The first encoder outputs the guided information and the side information for a decoder to reconstruct the source data.

公开(公告)号：US20220345704A1

公开(公告)日：2022-10-27

申请号：US17860585

申请日：2022-07-08

Applicant: GOOGLE LLC

Inventor： Sarah Parker ,  Debargha Mukherjee ,  Yue Chen ,  Elliott Karpilovsky ,  Urvang Joshi

IPC: H04N19/119 ,  H04N19/12 ,  G06N20/00 ,  H04N19/176 ,  H04N19/61

Abstract: Transform-level partitioning of a prediction residual block is performed to improve compression efficiency of video data. During encoding, a prediction residual block is generated responsive to prediction-level partitioning performed against a video block, a transform block partition type to use is determined based on the prediction residual block, a non-recursive transform-level partitioning is performed against the prediction residual block according to the transform block partition type, and transform blocks generated as a result of the transform-level partitioning are encoded to a bitstream. During decoding, a symbol representative of the transform block partition type used to encode transform blocks is derived from the bitstream, inverse transformed blocks are produced by inverse transforming encoded video data associated with the prediction residual block, and the prediction residual block is reproduced according to the transform block partition type and used to reconstruct the video block, which is output within an output video stream.

公开(公告)号：US11388401B2

公开(公告)日：2022-07-12

申请号：US16912767

申请日：2020-06-26

Applicant: GOOGLE LLC

Inventor： Sarah Parker ,  Debargha Mukherjee ,  Yue Chen ,  Elliott Karpilovsky ,  Urvang Joshi

IPC: H04N19/119 ,  H04N19/12 ,  G06N20/00 ,  H04N19/176 ,  H04N19/61

Abstract: Transform-level partitioning of a prediction residual block is performed to improve compression efficiency of video data. During encoding, a prediction residual block is generated responsive to prediction-level partitioning performed against a video block, a transform block partition type to use is determined based on the prediction residual block, a non-recursive transform-level partitioning is performed against the prediction residual block according to the transform block partition type, and transform blocks generated as a result of the transform-level partitioning are encoded to a bitstream. During decoding, a symbol representative of the transform block partition type used to encode transform blocks is derived from the bitstream, inverse transformed blocks are produced by inverse transforming encoded video data associated with the prediction residual block, and the prediction residual block is reproduced according to the transform block partition type and used to reconstruct the video block, which is output within an output video stream.

公开(公告)号：US11343528B2

公开(公告)日：2022-05-24

申请号：US17073892

申请日：2020-10-19

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  James Bankoski ,  Yue Chen ,  Yuxin Liu ,  Sarah Parker

IPC: H04N11/02 ,  H04N19/517 ,  H04N19/46 ,  H04N19/136 ,  H04N19/105 ,  H04N19/139 ,  H04N19/176 ,  H04N19/182

Abstract: Generating a compound predictor block of a current block of video can include generating, for the current block, predictor blocks comprising a first predictor block including first predictor pixels and a second predictor block including second predictor pixels; using at least a subset of the first predictor pixels to determine a first weight for a first predictor pixel of the first predictor pixels; obtaining a second weight for a second predictor pixel of the second predictor pixels, where the second predictor pixel is co-located with the first predictor pixel; and generating the compound predictor block by combining the first predictor block and the second predictor block, where the predictor block includes a weighted pixel that is determined using a weighted sum of the first predictor pixel and the second predictor pixel using the first weight and the second weight, respectively.

公开(公告)号：US20200260112A1

公开(公告)日：2020-08-13

申请号：US16861299

申请日：2020-04-29

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Emil Keyder ,  Michele Covell ,  Chen Wang ,  Sarah Parker ,  Ramin Zabih

IPC: H04N19/527 ,  H04N19/147 ,  H04N19/573 ,  H04N19/543 ,  H04N19/167 ,  H04N19/17 ,  H04N19/137 ,  H04N19/109 ,  H04N19/192 ,  H04N19/119 ,  H04N19/176 ,  H04N19/124 ,  H04N19/159 ,  H04N19/44 ,  H04N19/172 ,  G06T7/246

Abstract: An apparatus for encoding a current frame of a video. The apparatus includes a memory and a processor. The processor is configured to execute instructions stored in the memory to generate, for each reference frame of a subset of available reference frames, at least one respective candidate global motion model (GMM); partition the current frame into blocks; generate an aggregated residual frame for the current frame; and encode the respective residual blocks in a compressed bitstream. To generate the aggregated residual frame includes to select, for predicting each block of the blocks, a respective selected GMM, where the respective selected GMM corresponds to the one of the at least one respective candidate GMMs that minimizes a total error associated with the aggregated residual frame; and obtain respective residual blocks for the block.

公开(公告)号：US20180160117A1

公开(公告)日：2018-06-07

申请号：US15789400

申请日：2017-10-20

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Yue Chen ,  Sarah Parker

IPC: H04N19/117 ,  H04N19/895 ,  H04N19/86 ,  H04N19/154 ,  H04N19/176

Abstract: Restoring a degraded tile of a degraded frame resulting from reconstruction is disclosed. A method includes, for a scaling factor of at least some scaling factors, recursively filtering the degraded tile using the scaling factor to generate a respective restored tile, and determining a respective error for the respective restored tile with respect to the source tile. The method also includes selecting an optimal scaling factor from the at least some scaling factors and encoding, in an encoded bitstream, a scaling parameter based on the optimal scaling factor. The optimal scaling factor corresponding to a smallest respective error. An apparatus includes a processor and non-transitory memory storing instructions. The instructions cause the processor to determine, from an encoded bitstream, a scaling factor, which determines how strongly edges in the degraded tile affect filtering operations, and recursively filter, resulting in a restored tile, the degraded tile using the scaling factor.

公开(公告)号：US11282172B2

公开(公告)日：2022-03-22

申请号：US16515226

申请日：2019-07-18

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: G06T9/00 ,  G06T5/00 ,  G06T5/50 ,  G06N3/08 ,  G06N3/04 ,  H04N19/70 ,  H04N19/176 ,  G06N20/20 ,  G06T3/40 ,  G06K9/62

Abstract: Guided restoration is used to restore video data degraded from a video frame. The video frame is divided into restoration units (RUs) which each correspond to one or more blocks of the video frame. Restoration schemes are selected for each RU. The restoration schemes may indicate to use one of a plurality of neural networks trained for the guided restoration. Alternatively, the restoration schemes may indicate to use a neural network and a filter-based restoration tool. The video frame is then restored by processing each RU according to the respective selected restoration scheme. During encoding, the restored video frame is encoded to an output bitstream, and the use of the selected restoration schemes may be signaled within the output bitstream. During decoding, the restored video frame is output to an output video stream.

公开(公告)号：US20210037254A1

公开(公告)日：2021-02-04

申请号：US17073892

申请日：2020-10-19

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  James Bankoski ,  Yue Chen ,  Yuxin Liu ,  Sarah Parker

IPC: H04N19/517 ,  H04N19/46 ,  H04N19/136 ,  H04N19/105 ,  H04N19/139 ,  H04N19/176 ,  H04N19/182

Abstract: Generating a compound predictor block of a current block of video can include generating, for the current block, predictor blocks comprising a first predictor block including first predictor pixels and a second predictor block including second predictor pixels; using at least a subset of the first predictor pixels to determine a first weight for a first predictor pixel of the first predictor pixels; obtaining a second weight for a second predictor pixel of the second predictor pixels, where the second predictor pixel is co-located with the first predictor pixel; and generating the compound predictor block by combining the first predictor block and the second predictor block, where the predictor block includes a weighted pixel that is determined using a weighted sum of the first predictor pixel and the second predictor pixel using the first weight and the second weight, respectively.

公开(公告)号：US20200186796A1

公开(公告)日：2020-06-11

申请号：US16295176

申请日：2019-03-07

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: H04N19/107 ,  H04N19/105 ,  G06N20/00 ,  H04N19/184 ,  H04N19/154

Abstract: Video coding may include generating, by a processor, a decoded frame by decoding a current frame from an encoded bitstream and outputting a reconstructed frame based on the decoded frame. Decoding includes identifying a current encoded block from the current frame, identifying a prediction coding model for the current block, wherein the prediction coding model is a machine learning prediction coding model from a plurality of machine learning prediction coding models, identifying reference values for decoding the current block based on the prediction coding model, obtaining prediction values based on the prediction coding model and the reference values, generating a decoded block corresponding to the current encoded block based on the prediction values, and including the decoded block in the decoded frame.

公开(公告)号：US20200184603A1

公开(公告)日：2020-06-11

申请号：US16515226

申请日：2019-07-18

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: G06T5/00 ,  G06T5/50 ,  G06N3/08 ,  G06N3/04 ,  G06N20/20 ,  G06T9/00 ,  G06T3/40 ,  H04N19/70 ,  H04N19/176 ,  G06K9/62

Abstract: Guided restoration is used to restore video data degraded from a video frame. The video frame is divided into restoration units (RUs) which each correspond to one or more blocks of the video frame. Restoration schemes are selected for each RU. The restoration schemes may indicate to use one of a plurality of neural networks trained for the guided restoration. Alternatively, the restoration schemes may indicate to use a neural network and a filter-based restoration tool. The video frame is then restored by processing each RU according to the respective selected restoration scheme. During encoding, the restored video frame is encoded to an output bitstream, and the use of the selected restoration schemes may be signaled within the output bitstream. During decoding, the restored video frame is output to an output video stream.