-
公开(公告)号：US11699074B2
公开(公告)日：2023-07-11
申请号：US16746654
申请日：2020-01-17
Applicant: Google LLC
Inventor： Mohammad Norouzi , William Chan , Sara Sabour Rouh Aghdam
IPC: G06N3/08
CPC classification number: G06N3/08
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a sequence generation neural network. One of the methods includes obtaining a batch of training examples; for each of the training examples: processing the training network input in the training example using the neural network to generate an output sequence; for each particular output position in the output sequence: identifying a prefix that includes the system outputs at positions before the particular output position in the output sequence, for each possible system output in the vocabulary, determining a highest quality score that can be assigned to any candidate output sequence that includes the prefix followed by the possible system output, and determining an update to the current values of the network parameters that increases a likelihood that the neural network generates a system output at the position that has a high quality score.
-
公开(公告)号：US20220374658A1
公开(公告)日：2022-11-24
申请号：US17863070
申请日：2022-07-12
Applicant: Google LLC
Inventor： Ting Chen , Geoffrey Everest Hinton , Simon Kornblith , Mohammad Norouzi
Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.
-
公开(公告)号：US11386302B2
公开(公告)日：2022-07-12
申请号：US17018372
申请日：2020-09-11
Applicant: Google LLC
Inventor： Ting Chen , Simon Kornblith , Mohammad Norouzi , Geoffrey Everest Hinton , Kevin Jordan Swersky
IPC: G06V10/774 , G06K9/62 , G06N3/08
Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.
-
公开(公告)号：US20210256313A1
公开(公告)日：2021-08-19
申请号：US17180682
申请日：2021-02-19
Applicant: Google LLC
Inventor： Rishabh Agarwal , Chen Liang , Dale Eric Schuurmans , Mohammad Norouzi
Abstract: Methods and systems for learning policies using sparse and underspecified rewards. One of the methods includes training the policy jointly with an auxiliary reward function having a plurality of auxiliary reward parameters, the auxiliary reward function being configured to map, in accordance with the auxiliary reward parameters, trajectory features of at least a trajectory to an auxiliary reward value that indicates how well the trajectory performed a task in response to a context input.
-
公开(公告)号：US20200380023A1
公开(公告)日：2020-12-03
申请号：US16998891
申请日：2020-08-20
Applicant: Google LLC
Inventor： Gregory Sean Corrado , Tomas Mikolov , Samy Bengio , Yoram Singer , Jonathon Shlens , Andrea L. Frome , Jeffrey Adgate Dean , Mohammad Norouzi
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying data objects. One of the methods includes obtaining data that associates each term in a vocabulary of terms with a respective high-dimensional representation of the term; obtaining classification data for a data object, wherein the classification data includes a respective score for each of a plurality of categories, and wherein each of the categories is associated with a respective category label; computing an aggregate high-dimensional representation for the data object from high-dimensional representations for the category labels associated with the categories and the respective scores; identifying a first term in the vocabulary of terms having a high-dimensional representation that is closest to the aggregate high-dimensional representation; and selecting the first term as a category label for the data object.
-
Patent Agency Ranking
公开(公告)号：US20190130267A1
公开(公告)日：2019-05-02
申请号：US16174126
申请日：2018-10-29
Applicant: Google LLC
Inventor： Mohammad Norouzi , Daniel Aaron Abolafia , Quoc V. Le
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network using a priority queue. One of the methods includes maintaining data identifying a set of K output sequences that were previously generated; selecting at least one of the output sequences from the set of output sequences; for each selected output sequence, determining a respective score; determining, for each selected sequence, a respective first update to the current values of the controller parameters; generating a batch of new output sequences using the controller neural network; obtaining a respective reward for each of the new output sequences; determining, from the new output sequences and the output sequences in the maintained data, the K output sequences that have the highest rewards; and modifying the maintained data.
-
公开(公告)号：US20240338936A1
公开(公告)日：2024-10-10
申请号：US18296938
申请日：2023-04-06
Applicant: Google LLC
Inventor： Jonathan Ho , Tim Salimans , Alexey Alexeevich Gritsenko , William Chan , Mohammad Norouzi , David James Fleet
IPC: G06V10/82 , G06V10/771 , H04N7/01
CPC classification number: G06V10/82 , G06V10/771 , H04N7/0117 , H04N7/013
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output video conditioned on an input. In one aspect, a method comprises receiving the input; initializing a current intermediate representation; generating an output video by updating the current intermediate representation at each of a plurality of iterations, wherein the updating comprises, at each iteration: processing an intermediate input for the iteration comprising the current intermediate representation using a diffusion model that is configured to process the intermediate input to generate a noise output; and updating the current intermediate representation using the noise output for the iteration.
-
公开(公告)号：US20240220527A1
公开(公告)日：2024-07-04
申请号：US18606458
申请日：2024-03-15
Applicant: Google LLC
Inventor： Gregory Sean Corrado , Tomas Mikolov , Samuel Bengio , Yoram Singer , Jonathon Shlens , Andrea L. Frome , Jeffrey Adgate Dean , Mohammad Norouzi
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying data objects. One of the methods includes obtaining data that associates each term in a vocabulary of terms with a respective high-dimensional representation of the term; obtaining classification data for a data object, wherein the classification data includes a respective score for each of a plurality of categories, and wherein each of the categories is associated with a respective category label; computing an aggregate high-dimensional representation for the data object from high-dimensional representations for the category labels associated with the categories and the respective scores; identifying a first term in the vocabulary of terms having a high-dimensional representation that is closest to the aggregate high-dimensional representation; and selecting the first term as a category label for the data object.
-
公开(公告)号：US11978141B2
公开(公告)日：2024-05-07
申请号：US18199883
申请日：2023-05-19
Applicant: Google LLC
Inventor： Chitwan Saharia , William Chan , Mohammad Norouzi , Saurabh Saxena , Yi Li , Jay Ha Whang , David James Fleet , Jonathan Ho
IPC: G06T11/60 , G06F40/284 , G06F40/40 , G06N3/08 , G06T3/40 , G06T3/4053 , G06T5/00
CPC classification number: G06T11/60 , G06F40/284 , G06F40/40 , G06N3/08 , G06T3/4053 , G06T5/002
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating images. In one aspect, a method includes: receiving an input text prompt including a sequence of text tokens in a natural language; processing the input text prompt using a text encoder neural network to generate a set of contextual embeddings of the input text prompt; and processing the contextual embeddings through a sequence of generative neural networks to generate a final output image that depicts a scene that is described by the input text prompt.
-
公开(公告)号：US11847571B2
公开(公告)日：2023-12-19
申请号：US17863070
申请日：2022-07-12
Applicant: Google LLC
Inventor： Ting Chen , Geoffrey Everest Hinton , Simon Kornblith , Mohammad Norouzi
IPC: G06V10/00 , G06N3/084 , G06N3/08 , G06F18/21 , G06F18/241 , G06F18/214 , G06V10/764 , G06V10/774 , G06V10/778
CPC classification number: G06N3/084 , G06F18/2155 , G06F18/2178 , G06F18/241 , G06N3/08 , G06V10/764 , G06V10/7753 , G06V10/7788 , G06T2207/20081
Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.
-
-
-
-
-
-
-
-
-
Search Results
53
records
(took 0.014 seconds)
