公开(公告)号：US20200250515A1

公开(公告)日：2020-08-06

申请号：US16624949

申请日：2018-07-06

Applicant: Google LLC

Inventor： Ryan Rifkin ,  Ying Xiao ,  Shankar Krishnan

IPC: G06N3/04 ,  G06N3/08 ,  G06N20/00 ,  G06F17/16

Abstract: Generally, the present disclosure is directed to systems and methods for improved optimization of machine-learned models. In particular, the present disclosure provides stochastic optimization algorithms that are both faster than widely used algorithms for fixed amounts of computation, and are also able to scale up substantially better as more computational resources become available. The stochastic optimization algorithms can be used with large batch sizes. As an example, in some implementations, the systems and methods of the present disclosure can implicitly compute the inverse hessian of each mini-batch of training data to produce descent directions.