公开(公告)号：US20220092429A1

公开(公告)日：2022-03-24

申请号：US17481160

申请日：2021-09-21

Applicant: Google LLC

Inventor： Luke Shekerjian Metz ,  Niruban Maheswaranathan ,  Christian Daniel Freeman ,  Benjamin Poole ,  Jascha Narain Sohl-Dickstein

IPC: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes performing, using a plurality of training examples, a training step to obtain respective gradients of a loss function with respect to each of the parameters in the parameter tensors; obtaining a validation loss for a plurality of validation examples that are different from the plurality of training examples generating an optimizer input from at least the respective gradients and the validation loss; processing the optimizer input using an optimizer neural network to generate an output defining a respective update for each of the parameters in the parameter tensors of the neural network; and for each of the parameters in the parameter tensors, applying the respective update to a current value of the parameter to generate an updated value for the parameter.