Systems and Methods for Improved Optimization of Machine-Learned Models

    公开(公告)号:US20200250515A1

    公开(公告)日:2020-08-06

    申请号:US16624949

    申请日:2018-07-06

    Applicant: Google LLC

    Abstract: Generally, the present disclosure is directed to systems and methods for improved optimization of machine-learned models. In particular, the present disclosure provides stochastic optimization algorithms that are both faster than widely used algorithms for fixed amounts of computation, and are also able to scale up substantially better as more computational resources become available. The stochastic optimization algorithms can be used with large batch sizes. As an example, in some implementations, the systems and methods of the present disclosure can implicitly compute the inverse hessian of each mini-batch of training data to produce descent directions.

Patent Agency Ranking