公开(公告)号：US20230342616A1

公开(公告)日：2023-10-26

申请号：US18343579

申请日：2023-06-28

Applicant: Google LLC

Inventor： Ting Chen ,  Simon Komblith ,  Mohammad Norouzi ,  Geoffrey Everest Hinton ,  Kevin Jordan Swersky

IPC: G06N3/084 ,  G06F18/241 ,  G06F18/214 ,  G06V10/774 ,  G06V10/764 ,  G06V10/778 ,  G06N3/08 ,  G06F18/21

CPC classification number: G06N3/084 ,  G06F18/2155 ,  G06F18/2178 ,  G06F18/241 ,  G06N3/08 ,  G06V10/764 ,  G06V10/7753 ,  G06V10/7788 ,  G06T2207/20081

Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.

公开(公告)号：US12254413B2

公开(公告)日：2025-03-18

申请号：US18343579

申请日：2023-06-28

Applicant: Google LLC

Inventor： Ting Chen ,  Simon Komblith ,  Mohammad Norouzi ,  Geoffrey Everest Hinton ,  Kevin Jordan Swersky

IPC: G06V10/20 ,  G06F18/21 ,  G06F18/214 ,  G06F18/241 ,  G06N3/08 ,  G06N3/084 ,  G06V10/764 ,  G06V10/774 ,  G06V10/778

Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.

公开(公告)号：US20250086462A1

公开(公告)日：2025-03-13

申请号：US18960623

申请日：2024-11-26

Applicant: Google LLC

Inventor： Ting Chen ,  Simon Komblith ,  Mohammad Norouzi ,  Geoffrey Everest Hinton ,  Kevin Jordan Swersky

IPC: G06N3/084 ,  G06F18/21 ,  G06F18/214 ,  G06F18/241 ,  G06N3/08 ,  G06V10/764 ,  G06V10/774 ,  G06V10/778

Abstract: Systems, methods, and computer program products for performing semi-supervised contrastive learning of visual representations are provided. For example, the present disclosure provides systems and methods that leverage particular data augmentation schemes and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. Further, the present disclosure also provides improvements for semi-supervised contrastive learning. For example, computer-implemented method may include performing semi-supervised contrastive learning based on a set of one or more unlabeled training data, generating an image classification model based on a portion of a plurality of layers in a projection head neural network used in performing the contrastive learning, performing fine-tuning of the image classification model based on a set of one or more labeled training data, and after performing the fine-tuning, distilling the image classification model to a student model comprising a relatively smaller number of parameters than the image classification model.