公开(公告)号：US20190188568A1

公开(公告)日：2019-06-20

申请号：US15926768

申请日：2018-03-20

Applicant: salesforce.com, inc.

Inventor： Nitish Shirish KESKAR ,  Richard SOCHER

IPC: G06N3/08 ,  G06N3/04

CPC classification number: G06N3/082 ,  G06N3/0427

Abstract: Hybrid training of deep networks includes a multi-layer neural network. The training includes setting a current learning algorithm for the multi-layer neural network to a first learning algorithm. The training further includes iteratively applying training data to the neural network, determining a gradient for parameters of the neural network based on the applying of the training data, updating the parameters based on the current learning algorithm, and determining whether the current learning algorithm should be switched to a second learning algorithm based on the updating. The training further includes, in response to the determining that the current learning algorithm should be switched to a second learning algorithm, changing the current learning algorithm to the second learning algorithm and initializing a learning rate of the second learning algorithm based on the gradient and a step used by the first learning algorithm to update the parameters of the neural network.

公开(公告)号：US20210240943A1

公开(公告)日：2021-08-05

申请号：US17239297

申请日：2021-04-23

Applicant: salesforce.com, inc.

Inventor： Jasdeep SINGH ,  Nitish Shirish KESKAR ,  Bryan MCCANN

IPC: G06F40/58 ,  G06N20/00 ,  G06N3/08 ,  G06F40/51

Abstract: Approaches for cross-lingual regularization for multilingual generalization include a method for training a natural language processing (NLP) deep learning module. The method includes accessing a first dataset having a first training data entry, the first training data entry including one or more natural language input text strings in a first language; translating at least one of the one or more natural language input text strings of the first training data entry from the first language to a second language; creating a second training data entry by starting with the first training data entry and substituting the at least one of the natural language input text strings in the first language with the translation of the at least one of the natural language input text strings in the second language; adding the second training data entry to a second dataset; and training the deep learning module using the second dataset.