公开(公告)号：US20230004721A1

公开(公告)日：2023-01-05

申请号：US17655770

申请日：2022-03-21

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Shuai Zhang ,  Lijie Wang ,  Xinyan Xiao ,  Yue Chang

IPC: G06F40/30 ,  G06F40/211 ,  G06F40/284

Abstract: Disclosed are a method for training a semantic representation model, a device and a storage medium, which relate to the field of computer technologies, and particularly to the field of artificial intelligence, such as a natural language processing technology, a deep learning technology, or the like. The method for training a semantic representation model includes: obtaining an anchor sample based on a sentence, and obtaining a positive sample and a negative sample based on syntactic information of the sentence; processing the anchor sample, the positive sample and the negative sample using the semantic representation model respectively, so as to obtain an anchor-sample semantic representation, a positive-sample semantic representation and a negative-sample semantic representation; constructing a contrast loss function based on the anchor-sample semantic representation, the positive-sample semantic representation, and the negative-sample semantic representation; and training the semantic representation model based on the contrast loss function.

公开(公告)号：US11989516B2

公开(公告)日：2024-05-21

申请号：US17572068

申请日：2022-01-10

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Lijie Wang ,  Shuai Zhang ,  Xinyan Xiao ,  Yue Chang ,  Tingting Li

IPC: G06F40/289 ,  G06N20/00

CPC classification number: G06F40/289 ,  G06N20/00

Abstract: The present disclosure provides a method and apparatus for acquiring a pre-trained model, an electronic device and a storage medium, and relates to the field of artificial intelligence, such as the natural language processing field, the deep learning field, or the like. The method may include: adding, in a process of training a pre-trained model using training sentences, a learning objective corresponding to syntactic information for a self-attention module in the pre-trained model; and training the pre-trained model according to the defined learning objective. The solution of the present disclosure may improve a performance of the pre-trained model, and reduce consumption of computing resources, or the like.