公开(公告)号：US20200151567A1

公开(公告)日：2020-05-14

申请号：US16746654

申请日：2020-01-17

Applicant: Google LLC

Inventor： Mohammad Norouzi ,  William Chan ,  Sara Sabour Rouh Aghdam

IPC: G06N3/08 ,  G10L25/30 ,  G10L15/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a sequence generation neural network. One of the methods includes obtaining a batch of training examples; for each of the training examples: processing the training network input in the training example using the neural network to generate an output sequence; for each particular output position in the output sequence: identifying a prefix that includes the system outputs at positions before the particular output position in the output sequence, for each possible system output in the vocabulary, determining a highest quality score that can be assigned to any candidate output sequence that includes the prefix followed by the possible system output, and determining an update to the current values of the network parameters that increases a likelihood that the neural network generates a system output at the position that has a high quality score.

公开(公告)号：US20190362229A1

公开(公告)日：2019-11-28

申请号：US16421406

申请日：2019-05-23

Applicant: Google LLC

Inventor： Mohammad Norouzi ,  William Chan ,  Sara Sabour Rouh Aghdam

IPC: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a sequence generation neural network. One of the methods includes obtaining a batch of training examples; for each of the training examples: processing the training network input in the training example using the neural network to generate an output sequence; for each particular output position in the output sequence: identifying a prefix that includes the system outputs at positions before the particular output position in the output sequence, for each possible system output in the vocabulary, determining a highest quality score that can be assigned to any candidate output sequence that includes the prefix followed by the possible system output, and determining an update to the current values of the network parameters that increases a likelihood that the neural network generates a system output at the position that has a high quality score.

公开(公告)号：US20190188566A1

公开(公告)日：2019-06-20

申请号：US16328207

申请日：2017-08-25

Applicant: GOOGLE LLC

Inventor： Michael Schuster ,  Samuel Bengio ,  Navdeep Jaitly ,  Zhifeng Chen ,  Dale Eric Schuurmans ,  Mohammad Norouzi ,  Yonghui Wu

IPC: G06N3/08 ,  G06N20/00

CPC classification number: G06N3/08 ,  G06N20/00

Abstract: A method includes obtaining data identifying a machine learning model to be trained to perform a machine learning task, the machine learning model being configured to receive an input example and to process the input example in accordance with current values of a plurality of model parameters to generate a model output for the input example; obtaining initial training data for training the machine learning model, the initial training data comprising a plurality of training examples and, for each training example, a ground truth output that should be generated by the machine learning model by processing the training example; generating modified training data from the initial training data; and training the machine learning model on the modified training data.