公开(公告)号：US11803751B2

公开(公告)日：2023-10-31

申请号：US17140863

申请日：2021-01-04

申请人： Google LLC

发明人： Mohammad Saleh ,  Jingqing Zhang ,  Yao Zhao ,  Peter J. Liu

IPC分类号： G06N3/08 ,  G06F40/30 ,  G06N3/045

CPC分类号： G06N3/08 ,  G06F40/30 ,  G06N3/045

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text summarization neural network. One of the methods includes pre-training the text summarization neural network including learning values of a plurality of network parameters through self-supervised learning using unlabeled data comprising unlabeled first texts, the pre-training including: obtaining an unlabeled first text comprising a plurality of segments; selecting one or more of the plurality of segments; processing a masked first text that excludes the one or more selected segments to generate a prediction of the one or more selected segments; and determining, based on a difference between the prediction and the one or more selected segments, an update to the current values of the plurality of network parameters; adapting the pre-trained text summarization neural network for a specific text summarization task using labeled data comprising second texts and respective summaries of the second texts.

公开(公告)号：US20240185065A1

公开(公告)日：2024-06-06

申请号：US18485950

申请日：2023-10-12

申请人： Google LLC

发明人： Mohammad Saleh ,  Jingqing Zhang ,  Yao Zhao ,  Peter J. Liu

IPC分类号： G06N3/08 ,  G06F40/30 ,  G06N3/045

CPC分类号： G06N3/08 ,  G06F40/30 ,  G06N3/045

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text summarization neural network. One of the methods includes pre-training the text summarization neural network including learning values of a plurality of network parameters through self-supervised learning using unlabeled data comprising unlabeled first texts, the pre-training including: obtaining an unlabeled first text comprising a plurality of segments; selecting one or more of the plurality of segments; processing a masked first text that excludes the one or more selected segments to generate a prediction of the one or more selected segments; and determining, based on a difference between the prediction and the one or more selected segments, an update to the current values of the plurality of network parameters; adapting the pre-trained text summarization neural network for a specific text summarization task using labeled data comprising second texts and respective summaries of the second texts.

公开(公告)号：US10885436B1

公开(公告)日：2021-01-05

申请号：US16869419

申请日：2020-05-07

申请人： Google LLC

发明人： Mohammad Saleh ,  Jingqing Zhang ,  Yao Zhao ,  Peter J. Liu

IPC分类号： G06N3/08 ,  G06N3/04 ,  G06F40/30

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text summarization neural network. One of the methods includes pre-training the text summarization neural network including learning values of a plurality of network parameters through self-supervised learning using unlabeled data comprising unlabeled first texts, the pre-training including: obtaining an unlabeled first text comprising a plurality of segments; selecting one or more of the plurality of segments; processing a masked first text that excludes the one or more selected segments to generate a prediction of the one or more selected segments; and determining, based on a difference between the prediction and the one or more selected segments, an update to the current values of the plurality of network parameters; adapting the pre-trained text summarization neural network for a specific text summarization task using labeled data comprising second texts and respective summaries of the second texts.

公开(公告)号：US20210350229A1

公开(公告)日：2021-11-11

申请号：US17140863

申请日：2021-01-04

申请人： Google LLC

发明人： Mohammad Saleh ,  Jingqing Zhang ,  Yao Zhao ,  Peter J. Liu

IPC分类号： G06N3/08 ,  G06N3/04 ,  G06F40/30

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text summarization neural network. One of the methods includes pre-training the text summarization neural network including learning values of a plurality of network parameters through self-supervised learning using unlabeled data comprising unlabeled first texts, the pre-training including: obtaining an unlabeled first text comprising a plurality of segments; selecting one or more of the plurality of segments; processing a masked first text that excludes the one or more selected segments to generate a prediction of the one or more selected segments; and determining, based on a difference between the prediction and the one or more selected segments, an update to the current values of the plurality of network parameters; adapting the pre-trained text summarization neural network for a specific text summarization task using labeled data comprising second texts and respective summaries of the second texts.