公开(公告)号：US11989516B2

公开(公告)日：2024-05-21

申请号：US17572068

申请日：2022-01-10

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Lijie Wang ,  Shuai Zhang ,  Xinyan Xiao ,  Yue Chang ,  Tingting Li

IPC: G06F40/289 ,  G06N20/00

CPC classification number: G06F40/289 ,  G06N20/00

Abstract: The present disclosure provides a method and apparatus for acquiring a pre-trained model, an electronic device and a storage medium, and relates to the field of artificial intelligence, such as the natural language processing field, the deep learning field, or the like. The method may include: adding, in a process of training a pre-trained model using training sentences, a learning objective corresponding to syntactic information for a self-attention module in the pre-trained model; and training the pre-trained model according to the defined learning objective. The solution of the present disclosure may improve a performance of the pre-trained model, and reduce consumption of computing resources, or the like.

公开(公告)号：US20230004721A1

公开(公告)日：2023-01-05

申请号：US17655770

申请日：2022-03-21

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Shuai Zhang ,  Lijie Wang ,  Xinyan Xiao ,  Yue Chang

IPC: G06F40/30 ,  G06F40/211 ,  G06F40/284

Abstract: Disclosed are a method for training a semantic representation model, a device and a storage medium, which relate to the field of computer technologies, and particularly to the field of artificial intelligence, such as a natural language processing technology, a deep learning technology, or the like. The method for training a semantic representation model includes: obtaining an anchor sample based on a sentence, and obtaining a positive sample and a negative sample based on syntactic information of the sentence; processing the anchor sample, the positive sample and the negative sample using the semantic representation model respectively, so as to obtain an anchor-sample semantic representation, a positive-sample semantic representation and a negative-sample semantic representation; constructing a contrast loss function based on the anchor-sample semantic representation, the positive-sample semantic representation, and the negative-sample semantic representation; and training the semantic representation model based on the contrast loss function.

公开(公告)号：US12038955B2

公开(公告)日：2024-07-16

申请号：US17652314

申请日：2022-02-24

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Ao Zhang ,  Lijie Wang ,  Xinyan Xiao ,  Tingting Li

IPC: G06F16/00 ,  G06F16/33 ,  G06F16/332

CPC classification number: G06F16/3329 ,  G06F16/3347

Abstract: The disclosure provides a method for generating a query statement. The method includes: determining a first vector representation based on known nodes in a first syntax tree corresponding to a query statement to be generated; determining a target generation strategy corresponding to a target node to be generated based on the first vector representation and a preset copy reference matrix; generating the target node based on the first vector representation or a second vector representation by performing the target generation strategy, in which the second vector representation is a vector representation corresponding to an adjacent query statement prior to the query statement to be generated; and generating the query statement based on the known nodes and a terminator in response to the target node being the terminator.