专利检索 ap:("NAVER CORPORATION") AND inv:"Michael Niemaz" 第 1 页

1.

发明授权
Self-attention-based confidence estimation of language models 有权

公开(公告)号：US12124814B2

公开(公告)日：2024-10-22

申请号：US17720912

申请日：2022-04-14

申请人： NAVER CORPORATION

发明人： Julien Perez , Denys Proux , Michael Niemaz

IPC分类号： G06F16/9032 , G06F16/332 , G06F40/30 , G06F40/58 , G06N3/042 , G06N3/0455 , G06N3/0464 , G06N3/0895 , G06N3/10

CPC分类号： G06F40/58 , G06F16/90332 , G06N3/10

摘要： A confidence estimation system includes: a neural network including at least one an attention module including N heads configured to: generate attention matrices based on interactions between tokens for words in an input sequence of words, the input sequence of words including a word that is obscured; and determine the word that is obscured in the input sequence; and a confidence module configured to determine a confidence value indicative of a probability of the neural network correctly determining the word that is obscured, the confidence module determining the confidence value of the word that is obscured using a convolutional neural network that projects the attention matrices generated by the attention module over a multi-dimensional space, the attention matrices recording interactions between the tokens in the input sequence of words without information regarding the tokens for the words and the word that is obscured.

2.

发明公开
SELF-ATTENTION-BASED CONFIDENCE ESTIMATION OF LANGUAGE MODELS 审中-公开

公开(公告)号：US20230334267A1

公开(公告)日：2023-10-19

申请号：US17720912

申请日：2022-04-14

申请人： NAVER CORPORATION

发明人： Julien PEREZ , Denys Proux , Michael Niemaz

IPC分类号： G06F40/58 , G06N3/10 , G06F16/9032

CPC分类号： G06F40/58 , G06F16/90332 , G06N3/10

摘要： A confidence estimation system includes: a neural network including at least one an attention module including N heads configured to: generate attention matrices based on interactions between tokens for words in an input sequence of words, the input sequence of words including a word that is obscured; and determine the word that is obscured in the input sequence; and a confidence module configured to determine a confidence value indicative of a probability of the neural network correctly determining the word that is obscured, the confidence module determining the confidence value of the word that is obscured using a convolutional neural network that projects the attention matrices generated by the attention module over a multi-dimensional space, the attention matrices recording interactions between the tokens in the input sequence of words without information regarding the tokens for the words and the word that is obscured.