公开(公告)号：US20170270407A1

公开(公告)日：2017-09-21

申请号：US15407470

申请日：2017-01-17

Applicant: Google Inc.

Inventor： Christopher Alberti ,  Aliaksei Severyn ,  Daniel Andor ,  Slav Petrov ,  Kuzman Ganchev Ganchev ,  David Joseph Weiss ,  Michael John Collins ,  Alessandro Presta

IPC: G06N3/08 ,  G06N3/04

CPC classification number: G06N3/08 ,  G06N3/04 ,  G06N5/003 ,  G06N7/005

Abstract: A method includes training a neural network having parameters on training data, in which the neural network receives an input state and processes the input state to generate a respective score for each decision in a set of decisions. The method includes receiving training data including training text sequences and, for each training text sequence, a corresponding gold decision sequence. The method includes training the neural network on the training data to determine trained values of parameters of the neural network. Training the neural network includes for each training text sequence: maintaining a beam of candidate decision sequences for the training text sequence, updating each candidate decision sequence by adding one decision at a time, determining that a gold candidate decision sequence matching a prefix of the gold decision sequence has dropped out of the beam, and in response, performing an iteration of gradient descent to optimize an objective function.