公开(公告)号：US20210158211A1

公开(公告)日：2021-05-27

申请号：US16953977

申请日：2020-11-20

Applicant: Google LLC

Inventor： Kunal Talwar ,  Vitaly Feldman ,  Tomer Koren

IPC: G06N20/00 ,  G06F17/18 ,  G06F21/60

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for training a machine learning model. The method includes obtaining a training data set comprising a plurality of training examples; determining i) a stochastic gradient descent step size schedule, ii) a stochastic gradient descent noise schedule, and iii) a stochastic gradient descent batch size schedule, wherein the stochastic gradient descent batch size schedule comprises a sequence of varying batch sizes; and training a machine learning model on the training data set, comprising performing stochastic gradient descent according to the i) stochastic gradient descent step size schedule, ii) stochastic gradient descent noise schedule, and iii) stochastic gradient descent batch size schedule to adjust a machine learning model loss function.