公开(公告)号：US20240256862A1

公开(公告)日：2024-08-01

申请号：US18424689

申请日：2024-01-26

Applicant: Google LLC

Inventor： Ting Chen

IPC: G06N3/08 ,  G06N3/048

CPC classification number: G06N3/08 ,  G06N3/048

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a network output using a diffusion neural network and for training a diffusion neural network with a modified noise scheduling strategy.

公开(公告)号：US20230342616A1

公开(公告)日：2023-10-26

申请号：US18343579

申请日：2023-06-28

Applicant: Google LLC

Inventor： Ting Chen ,  Simon Komblith ,  Mohammad Norouzi ,  Geoffrey Everest Hinton ,  Kevin Jordan Swersky

IPC: G06N3/084 ,  G06F18/241 ,  G06F18/214 ,  G06V10/774 ,  G06V10/764 ,  G06V10/778 ,  G06N3/08 ,  G06F18/21

CPC classification number: G06N3/084 ,  G06F18/2155 ,  G06F18/2178 ,  G06F18/241 ,  G06N3/08 ,  G06V10/764 ,  G06V10/7753 ,  G06V10/7788 ,  G06T2207/20081

Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.

公开(公告)号：US20220374658A1

公开(公告)日：2022-11-24

申请号：US17863070

申请日：2022-07-12

Applicant: Google LLC

Inventor： Ting Chen ,  Geoffrey Everest Hinton ,  Simon Kornblith ,  Mohammad Norouzi

IPC: G06K9/62 ,  G06N3/08

Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.

公开(公告)号：US11386302B2

公开(公告)日：2022-07-12

申请号：US17018372

申请日：2020-09-11

Applicant: Google LLC

Inventor： Ting Chen ,  Simon Kornblith ,  Mohammad Norouzi ,  Geoffrey Everest Hinton ,  Kevin Jordan Swersky

IPC: G06V10/774 ,  G06K9/62 ,  G06N3/08

Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.