公开(公告)号：US11900235B1

公开(公告)日：2024-02-13

申请号：US17470716

申请日：2021-09-09

Applicant: Google LLC

Inventor： Andrew M. Dai ,  Quoc V. Le ,  David Ha

IPC: G06N3/04 ,  G06N3/08

CPC classification number: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using recurrent neural networks. One of the systems includes a main recurrent neural network comprising one or more recurrent neural network layers and a respective hyper recurrent neural network corresponding to each of the one or more recurrent neural network layers, wherein each hyper recurrent neural network is configured to, at each of a plurality of time steps: process the layer input at the time step to the corresponding recurrent neural network layer, the current layer hidden state of the corresponding recurrent neural network layer, and a current hypernetwork hidden state of the hyper recurrent neural network to generate an updated hypernetwork hidden state.

公开(公告)号：US11164066B1

公开(公告)日：2021-11-02

申请号：US15716330

申请日：2017-09-26

Applicant: Google LLC

Inventor： Andrew M. Dai ,  Quoc V. Le ,  David Ha

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using recurrent neural networks. One of the systems includes a main recurrent neural network comprising one or more recurrent neural network layers and a respective hyper recurrent neural network corresponding to each of the one or more recurrent neural network layers, wherein each hyper recurrent neural network is configured to, at each of a plurality of time steps: process the layer input at the time step to the corresponding recurrent neural network layer, the current layer hidden state of the corresponding recurrent neural network layer, and a current hypernetwork hidden state of the hyper recurrent neural network to generate an updated hypernetwork hidden state.

公开(公告)号：US20190354868A1

公开(公告)日：2019-11-21

申请号：US16526240

申请日：2019-07-30

Applicant: Google LLC

Inventor： Daniel Pieter Wierstra ,  Chrisantha Thomas Fernando ,  Alexander Pritzel ,  Dylan Sunil Banarse ,  Charles Blundell ,  Andrei-Alexandru Rusu ,  Yori Zwols ,  David Ha

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using multi-task neural networks. One of the methods includes receiving a first network input and data identifying a first machine learning task to be performed on the first network input; selecting a path through the plurality of layers in a super neural network that is specific to the first machine learning task, the path specifying, for each of the layers, a proper subset of the modular neural networks in the layer that are designated as active when performing the first machine learning task; and causing the super neural network to process the first network input using (i) for each layer, the modular neural networks in the layer that are designated as active by the selected path and (ii) the set of one or more output layers corresponding to the identified first machine learning task.