Invention Grant
- Patent Title: Pointer sentinel mixture architecture
-
Application No.: US16664508Application Date: 2019-10-25
-
Publication No.: US11580359B2Publication Date: 2023-02-14
- Inventor: Stephen Joseph Merity , Caiming Xiong , James Bradbury , Richard Socher
- Applicant: salesforce.com, inc.
- Applicant Address: US CA San Francisco
- Assignee: salesforce.com, inc.
- Current Assignee: salesforce.com, inc.
- Current Assignee Address: US CA San Francisco
- Agency: Haynes and Boone LLP
- Main IPC: G06N3/04
- IPC: G06N3/04 ; G06N3/084 ; G06F40/284 ; G06N3/08 ; G06N7/00

Abstract:
The technology disclosed provides a so-called “pointer sentinel mixture architecture” for neural network sequence models that has the ability to either reproduce a token from a recent context or produce a token from a predefined vocabulary. In one implementation, a pointer sentinel-LSTM architecture achieves state of the art language modeling performance of 70.9 perplexity on the Penn Treebank dataset, while using far fewer parameters than a standard softmax LSTM.
Public/Granted literature
- US20200065651A1 POINTER SENTINEL MIXTURE ARCHITECTURE Public/Granted day:2020-02-27
Information query