Invention Grant
- Patent Title: Hybrid training of deep networks
-
Application No.: US15926768Application Date: 2018-03-20
-
Publication No.: US11276002B2Publication Date: 2022-03-15
- Inventor: Nitish Shirish Keskar , Richard Socher
- Applicant: salesforce.com, inc.
- Applicant Address: US CA San Francisco
- Assignee: salesforce.com, inc.
- Current Assignee: salesforce.com, inc.
- Current Assignee Address: US CA San Francisco
- Agency: Haynes and Boone, LLP
- Main IPC: G06N3/08
- IPC: G06N3/08 ; G06N3/04

Abstract:
Hybrid training of deep networks includes a multi-layer neural network. The training includes setting a current learning algorithm for the multi-layer neural network to a first learning algorithm. The training further includes iteratively applying training data to the neural network, determining a gradient for parameters of the neural network based on the applying of the training data, updating the parameters based on the current learning algorithm, and determining whether the current learning algorithm should be switched to a second learning algorithm based on the updating. The training further includes, in response to the determining that the current learning algorithm should be switched to a second learning algorithm, changing the current learning algorithm to the second learning algorithm and initializing a learning rate of the second learning algorithm based on the gradient and a step used by the first learning algorithm to update the parameters of the neural network.
Public/Granted literature
- US20190188568A1 HYBRID TRAINING OF DEEP NETWORKS Public/Granted day:2019-06-20
Information query