公开(公告)号：US20250124537A1

公开(公告)日：2025-04-17

申请号：US18999336

申请日：2024-12-23

Applicant: Google LLC

Inventor： Junjie Ke ,  Feng Yang ,  Qifei Wang ,  Yilin Wang ,  Peyman Milanfar

IPC: G06T3/04 ,  G06T3/40 ,  G06T7/00

Abstract: The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

公开(公告)号：US12250383B2

公开(公告)日：2025-03-11

申请号：US17911245

申请日：2020-05-19

Applicant: Google LLC

Inventor： Yilin Wang ,  Balineedu Adsumilli

IPC: H04N19/00 ,  H04N19/149 ,  H04N19/154 ,  H04N21/2743

Abstract: Video streams uploaded to a video hosting platform are transcoded using quality-normalized transcoding parameters dynamically selected using a learning model. Video frames of a video stream are processed using the learning model to determine bitrate and quality score pairs for some or all possible transcoding resolutions. The listing of bitrate and quality score pairs determined for each resolution is processed to determine a set of transcoding parameters for transcoding the video stream into each resolution. The bitrate and quality score pairs of a given listing may be processed using one or more predefined thresholds, which may, in some cases, refer to a weighted distribution of resolutions according to watch times of videos of the video hosting platform. The video stream is then transcoded into the various resolutions using the set of transcoding parameters selected for each resolution.

公开(公告)号：US20240422369A1

公开(公告)日：2024-12-19

申请号：US18336577

申请日：2023-06-16

Applicant: Google LLC

Inventor： Yilin Wang ,  Miao Yin ,  Qifei Wang ,  Boqing Gong ,  Neil Aylon Charles Birkbeck ,  Balineedu Chowdary Adsumilli

IPC: H04N21/2343 ,  G06T7/00 ,  H04N19/132 ,  H04N21/466 ,  H04N21/485

Abstract: A method for generating, for a video stream of a first spatial resolution and a first temporal resolution, a first reduced quality steam of a second spatial resolution and a second reduced-quality stream of a second temporal resolution. A first subset of STPs is sampled from the first reduced-quality stream and a second subset of STPs is sampled from the second reduced-quality stream. Using a machine learning model (MLM) the STPs are processed to identify a quality score for each quality-representative STPs that are representative of a quality of the video stream. One or more quality-improving actions for the video stream are identified using the quality scores of the quality-representative STPs.

公开(公告)号：US20230319327A1

公开(公告)日：2023-10-05

申请号：US18021636

申请日：2022-06-08

Applicant: Google LLC

Inventor： Yilin Wang ,  Balineedu Adsumilli ,  Junjie Ke ,  Hossein Talebi ,  Joong Yim ,  Neil Birkbeck ,  Peyman Milanfar ,  Feng Yang

IPC: H04N21/234 ,  H04N19/154 ,  H04N21/466

CPC classification number: H04N21/23418 ,  H04N19/154 ,  H04N21/4668

Abstract: Methods, systems, and media for determining perceptual quality indicators of video content items are provided. In some embodiments, the method comprises: receiving a video content item; extracting a plurality of frames from the video content item; determining, using a first subnetwork of a deep neural network, a content quality indicator for each frame of the plurality of frames of the video content item; determining, using a second subnetwork of the deep neural network, a video distortion indicator for each frame of the plurality of frames of the video content item; determining, using a third subnetwork of the deep neural network, a compression sensitivity indicator for each frame of the plurality of frames of the video content item; generating a quality level for each frame of the plurality of frames of the video content item that concatenates the content quality indicator, the video distortion indicator, and the compression sensitivity indicator for that frame of the video content item; generating an overall quality level for video content item by aggregating the quality level of each frame of the plurality of frames; and causing a video recommendation to be presented based on the overall quality level of the video content item.

公开(公告)号：US12273521B2

公开(公告)日：2025-04-08

申请号：US17862571

申请日：2022-07-12

Applicant: GOOGLE LLC

Inventor： Yilin Wang ,  Balineedu Adsumilli

IPC: H04N19/13 ,  G06F18/214 ,  G06N20/00 ,  G06T7/00

Abstract: A training dataset that includes a first dataset and a second dataset is received. The first dataset includes a first subset of first videos corresponding to a first context and respective first ground truth quality scores of the first videos, and the second dataset includes a second subset of second videos corresponding to a second context and respective second ground truth quality scores of the second videos. A machine learning model is trained to predict the respective first ground truth quality scores and the respective second ground truth quality scores. Training the model includes training it to obtain a global quality score for one of the videos; and training it to map the global quality score to context-dependent predicted quality scores. The context-dependent predicted quality scores include a first context-dependent predicted quality score corresponding to the first context and a second context-dependent predicted quality score corresponding to the second context.

公开(公告)号：US20250071299A1

公开(公告)日：2025-02-27

申请号：US18455298

申请日：2023-08-24

Applicant: GOOGLE LLC

Inventor： Yao-Chung Lin ,  Jingning Han ,  Yilin Wang ,  Yeping Su

IPC: H04N19/196 ,  H04N19/105 ,  H04N19/176 ,  H04N19/80

Abstract: Encoding using media compression and processing for machine-learning-based quality metrics includes generating encoded frame data by encoding a current frame from an input video using a neural-network-based video quality model, which includes identifying optimal encoding parameters for encoding a current block, wherein the optimal encoding parameters minimize a rate-distortion optimization cost function, which includes using a gradient value for the current block obtained from a neural-network-based video quality model generated gradient map obtained from the neural-network-based video quality model for the current frame, obtaining a restoration filtered reconstructed frame by restoration filtering a reconstructed frame, obtained by decoding the encoded frame data, using the neural-network-based video quality model generated gradient map obtained for the reconstructed frame.

公开(公告)号：US11854165B2

公开(公告)日：2023-12-26

申请号：US17922531

申请日：2020-05-19

Applicant: Google LLC

Inventor： Yilin Wang ,  Balineedu Adsumilli ,  Feng Yang

IPC: G06T5/00 ,  G06T5/20 ,  G06V10/56 ,  G06V10/74 ,  G06T7/13

CPC classification number: G06T5/002 ,  G06T5/20 ,  G06T7/13 ,  G06V10/56 ,  G06V10/761 ,  G06T2207/20081

Abstract: A method includes training a first model to measure the banding artefacts, training a second model to deband the image, and generating a debanded image for the image using the second model. Training the first model can include selecting a first set of first training images, generating a banding edge map for a first training image, where the map includes weights that emphasize banding edges and de-emphasize true edges in the first training image, and using the map and a luminance plane of the first training image as input to the first model. Training the second model can include selecting a second set of second training images, generating a debanded training image for a second training image, generating a banding score for the debanded training image using the first model, and using the banding score in a loss function used in training the second model.

公开(公告)号：US10999578B2

公开(公告)日：2021-05-04

申请号：US16612889

申请日：2017-12-12

Applicant: Google LLC

Inventor： Chao Chen ,  Balineedu Adsumilli ,  Shawn Singh ,  Yilin Wang

IPC: H04N19/124 ,  H04N19/103 ,  H04N19/154 ,  H04N19/172 ,  H04N19/176 ,  H04N19/61

Abstract: Systems and methods for transcoding media content are disclosed. In some embodiments, the method includes obtaining a transcoded media content that is transcoded from an uploaded media content. The method includes determining a plurality of degradation metric values corresponding to the transcoded media content based on the uploaded media content and the transcoded media content, each degradation metric value corresponding to a different degradation metric type. The method includes mapping each degradation metric value to a respective calibrated score to obtain a plurality of calibrated scores. The method includes determining an aggregated quality score of the transcoded media content based on the plurality of calibrated scores and an exponential weighting function. The exponential weighting function exponentiates each of the calibrated scores by a respective weighting exponent and aggregates the exponentiated calibrated scores. The method includes selectively reencoding the transcoded media content based on the aggregated quality score.