Invention Application
- Patent Title: SPARSE ATTENTION NEURAL NETWORKS
-
Application No.: US17666400Application Date: 2022-02-07
-
Publication No.: US20220253672A1Publication Date: 2022-08-11
- Inventor: Aakanksha Chowdhery , Afroz Mohiuddin , Henryk Michalewski , Jonni Miikka Kanerva , Lukasz Mieczyslaw Kaiser , Sebastian Dariusz Jaszczur , Wojciech Gajewski
- Applicant: Google LLC
- Applicant Address: US CA Mountain View
- Assignee: Google LLC
- Current Assignee: Google LLC
- Current Assignee Address: US CA Mountain View
- Main IPC: G06N3/04
- IPC: G06N3/04

Abstract:
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a neural network configured to perform the machine learning task, the neural network including one or more sparse attention layers.
Information query