公开(公告)号：US20220101624A1

公开(公告)日：2022-03-31

申请号：US17423612

申请日：2020-01-22

Applicant: Google LLC

Inventor： Geoffrey E. Hinton ,  Nicholas Myles Wisener Frosst ,  Nicolas Guy Robert Papernot

IPC: G06V10/774 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a classification neural network. In one aspect, a method comprises: for each of a plurality of network inputs: processing the network input using the classification neural network to generate a classification output that defines a predicted class of the network input; determining a soft nearest neighbor loss, wherein the soft nearest neighbor loss encourages intermediate representations of network inputs of different classes to become more entangled, wherein the entanglement of intermediate representations of network inputs of different classes characterizes how similar pairs of intermediate representations of network inputs of different class are relative to pairs of intermediate representations of network inputs of the same class; and adjusting the current values of the classification neural network parameters using gradients of the soft nearest neighbor loss with respect to the classification neural network parameters.

公开(公告)号：US11941867B2

公开(公告)日：2024-03-26

申请号：US17423612

申请日：2020-01-22

Applicant: Google LLC

Inventor： Geoffrey E. Hinton ,  Nicholas Myles Wisener Frosst ,  Nicolas Guy Robert Papernot

IPC: G06K9/62 ,  G06N3/045 ,  G06N3/047 ,  G06V10/774

CPC classification number: G06V10/774 ,  G06N3/045 ,  G06N3/047

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a classification neural network. In one aspect, a method comprises: for each of a plurality of network inputs: processing the network input using the classification neural network to generate a classification output that defines a predicted class of the network input; determining a soft nearest neighbor loss, wherein the soft nearest neighbor loss encourages intermediate representations of network inputs of different classes to become more entangled, wherein the entanglement of intermediate representations of network inputs of different classes characterizes how similar pairs of intermediate representations of network inputs of different class are relative to pairs of intermediate representations of network inputs of the same class; and adjusting the current values of the classification neural network parameters using gradients of the soft nearest neighbor loss with respect to the classification neural network parameters.