公开(公告)号：US12204851B2

公开(公告)日：2025-01-21

申请号：US17864636

申请日：2022-07-14

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Tongyang Liu ,  Shu Wang ,  Wanli Chang ,  Wei Zheng ,  Zhifan Feng ,  Chunguang Chai ,  Yong Zhu

IPC: G06F40/211 ,  G06F40/109 ,  G06F40/30 ,  G06N3/08

Abstract: A method for generating a pre-trained language model, includes: obtaining sample files; obtaining typography structure information and text information of the sample files by parsing the sample files; obtaining a plurality of task models of a pre-trained language model; obtaining a trained pre-trained language model by jointly training the pre-trained language model and the plurality of task models according to the typography structure information and the text information; and generating a target pre-trained language model by fine-tuning the trained pre-trained language model according to the typography structure information and the text information.