-
公开(公告)号:US20230099526A1
公开(公告)日:2023-03-30
申请号:US17486359
申请日:2021-09-27
Applicant: Google LLC
Inventor: Troy Chinen , Alex Sukhanov , Eirikur Thor Agustsson , George Dan Toderici
Abstract: Example aspects of the present disclosure are directed to a computer-implemented method for determining a perceptual quality of a subject video content item. The method can include inputting a subject frame set from the subject video content item into a first machine-learned model. The method can also include generating, using the first machine-learned model, a feature based at least in part on the subject frame set. The method can also include outputting, using a second machine-learned model, a score indicating the perceptual quality of the subject video content item based at least in part on the feature.
-
公开(公告)号:US11610124B2
公开(公告)日:2023-03-21
申请号:US16666689
申请日:2019-10-29
Applicant: Google LLC
Inventor: Abhinav Shrivastava , Saurabh Singh , Johannes Balle , Sami Ahmad Abu-El-Haija , Nicholas Johnston , George Dan Toderici
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving, by a neural network (NN), a dataset for generating features from the dataset. A first set of features is computed from the dataset using at least a feature layer of the NN. The first set of features i) is characterized by a measure of informativeness; and ii) is computed such that a size of the first set of features is compressible into a second set of features that is smaller in size than the first set of features and that has a same measure of informativeness as the measure of informativeness of the first set of features. The second set of features if generated from the first set of features using a compression method that compresses the first set of features to generate the second set of features.
-
公开(公告)号:US11257254B2
公开(公告)日:2022-02-22
申请号:US16515586
申请日:2019-07-18
Applicant: Google LLC
Inventor: David Charles Minnen , Saurabh Singh , Johannes Balle , Troy Chinen , Sung Jin Hwang , Nicholas Johnston , George Dan Toderici
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for compressing and decompressing data. In one aspect, a method comprises: processing data using an encoder neural network to generate a latent representation of the data; processing the latent representation of the data using a hyper-encoder neural network to generate a latent representation of an entropy model; generating an entropy encoded representation of the latent representation of the entropy model; generating an entropy encoded representation of the latent representation of the data using the latent representation of the entropy model; and determining a compressed representation of the data from the entropy encoded representations of: (i) the latent representation of the data and (ii) the latent representation of the entropy model used to entropy encode the latent representation of the data.
-
公开(公告)号:US11074454B1
公开(公告)日:2021-07-27
申请号:US16410863
申请日:2019-05-13
Applicant: Google LLC
Inventor: Sudheendra Vijayanarasimhan , George Dan Toderici , Yue Hei Ng , Matthew John Hausknecht , Oriol Vinyals , Rajat Monga
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying videos using neural networks. One of the methods includes obtaining a temporal sequence of video frames, wherein the temporal sequence comprises a respective video frame from a particular video at each of a plurality time steps; for each time step of the plurality of time steps: processing the video frame at the time step using a convolutional neural network to generate features of the video frame; and processing the features of the video frame using an LSTM neural network to generate a set of label scores for the time step and classifying the video as relating to one or more of the topics represented by labels in the set of labels from the label scores for each of the plurality of time steps.
-
公开(公告)号:US20180239964A1
公开(公告)日:2018-08-23
申请号:US15959858
申请日:2018-04-23
Applicant: Google LLC
Inventor: Sanketh Shetty , Tomas Izo , Min-Hsuan Tsai , Sudheendra Vijayanarasimhan , Apostol Natsev , Sami Abu-El-Haija , George Dan Toderici , Susanna Ricco , Balakrishnan Varadarajan , Nicola Muscettola , WeiHsin Gu , Weilong Yang , Nitin Khandelwal , Phuong Le
CPC classification number: G06K9/00718 , G06F16/7834 , G06K9/00744 , G06K9/00751 , G06K9/00765 , G06K2209/27
Abstract: A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.
-
公开(公告)号:US09953222B2
公开(公告)日:2018-04-24
申请号:US14848216
申请日:2015-09-08
Applicant: Google LLC
Inventor: Sanketh Shetty , Tomas Izo , Min-Hsuan Tsai , Sudheendra Vijayanarasimhan , Apostol Natsev , Sami Abu-El-Haija , George Dan Toderici , Susanna Ricco , Balakrishnan Varadarajan , Nicola Muscettola , WeiHsin Gu , Weilong Yang , Nitin Khandelwal , Phuong Le
CPC classification number: G06K9/00718 , G06F17/30787 , G06K9/00744 , G06K9/00751 , G06K9/00765 , G06K2209/27
Abstract: A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.
-
公开(公告)号:US20240223817A1
公开(公告)日:2024-07-04
申请号:US18563734
申请日:2022-07-05
Applicant: Google LLC
Inventor: George Dan Toderici , Eirikur Thor Agustsson , Fabian Julius Mentzer , David Charles Minnen , Johannes Balle , Nicholas Johnston
IPC: H04N19/91 , G06T3/18 , G06T5/70 , H04N19/124 , H04N19/137
CPC classification number: H04N19/91 , G06T3/18 , G06T5/70 , H04N19/124 , H04N19/137 , G06T2207/20084
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for compressing video data. In one aspect, a method comprises: receiving a video sequence of frames; generating, using a flow prediction network, an optical flow between two sequential frames, wherein the two sequential frames comprise a first frame and a second frame that is subsequent the first frame; generating from the optical flow, using a first autoencoder neural network: a predicted optical flow between the first frame and the second frame; and warping a reconstruction of the first frame according to the predicted optical flow and subsequently applying a blurring operation to obtain an initial predicted reconstruction of the second frame.
-
公开(公告)号:US20240107079A1
公开(公告)日:2024-03-28
申请号:US18238068
申请日:2023-08-25
Applicant: Google LLC
Inventor: George Dan Toderici , Fabian Julius Mentzer , Eirikur Thor Agustsson , Michael Tobias Tschannen
IPC: H04N19/91 , G06N3/045 , G06N3/088 , H04N19/124 , H04N19/154
CPC classification number: H04N19/91 , G06N3/045 , G06N3/088 , H04N19/124 , H04N19/154
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an encoder neural network configured to receive a data item and to process the data item to output a compressed representation of the data item. In one aspect, a method includes, for each training data item: processing the data item using the encoder neural network to generate a latent representation of the training data item; processing the latent representation using a hyper-encoder neural network to determine a conditional entropy model; generating a compressed representation of the training data item; processing the compressed representation using a decoder neural network to generate a reconstruction of the training data item; processing the reconstruction of the training data item using a discriminator neural network to generate a discriminator network output; evaluating a first loss function; and determining an update to the current values of the encoder network parameters.
-
公开(公告)号:US20230260197A1
公开(公告)日:2023-08-17
申请号:US17708628
申请日:2022-03-30
Applicant: Google LLC
Inventor: Philip Andrew Chou , Berivan Isik , Sung Jin Hwang , Nicholas Milo Johnston , George Dan Toderici
IPC: G06T15/08 , H04N19/46 , H04N19/176
CPC classification number: G06T15/08 , H04N19/46 , H04N19/176
Abstract: Example embodiments of the present disclosure relate to systems and methods for compressing attributes of volumetric and hypervolumetric datasets. An example system performs operations including obtaining a reference dataset comprising attributes indexed by a domain of multidimensional coordinates; subdividing the domain into a plurality of blocks respectively associated with a plurality of attribute subsets; inputting, to a local nonlinear operator, a latent representation for an attribute subset associated with at least one block of the plurality of blocks; obtaining, using the local nonlinear operator and based on the latent representation, an attribute representation of one or more attributes of the attribute subset; and updating the latent representation based on a comparison of the attribute representation and the reference dataset.
-
公开(公告)号:US11354822B2
公开(公告)日:2022-06-07
申请号:US16610063
申请日:2018-05-16
Applicant: GOOGLE LLC
Inventor: Michele Covell , Damien Vincent , David Charles Minnen , Saurabh Singh , Sung Jin Hwang , Nicholas Johnston , Joel Eric Shor , George Dan Toderici
IPC: G06T9/00
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for image compression and reconstruction. A request to generate an encoded representation of an input image is received. The encoded representation of the input image is then generated. The encoded representation includes a respective set of binary codes at each iteration. Generating the set of binary codes for the iteration from an initial set of binary includes: for any tiles that have already been masked off during any previous iteration, masking off the tile. For any tiles that have not yet been masked off during any of the previous iterations, a determination is made as to whether a reconstruction error of the tile when reconstructed from binary codes at the previous iterations satisfies an error threshold. When the reconstruction quality satisfies the error threshold, the tile is masked off.
-
-
-
-
-
-
-
-
-