公开(公告)号：US20220129753A1

公开(公告)日：2022-04-28

申请号：US17572921

申请日：2022-01-11

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Yuxiang LU ,  Jiaxiang LIU ,  Xuyi CHEN ,  Shikun FENG ,  Shuohuan WANG ,  Yu SUN ,  Shiwei HUANG ,  Jingzhou HE

IPC: G06N3/08 ,  G06N3/04

Abstract: A pre-training method of a neural network model, an electronic device, and a medium. The pre-training data is inputted to the initial neural network model, and the initial neural network model is pre-trained in the first training mode, in the first training mode, the plurality of hidden layers share one hidden layer parameter, and the loss value of the initial neural network model is obtained, if the loss value of the initial neural network model is less than a preset threshold, the initial neural network model continues to be pre-trained in the second training mode, in the second training mode, each of the plurality of hidden layers has its own hidden layer parameter.

公开(公告)号：US20230047980A1

公开(公告)日：2023-02-16

申请号：US17976049

申请日：2022-10-28

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Xuyi CHEN ,  Weixin LIU ,  Yuxiang LU ,  Jiaxiang LU ,  Shiwei HUANG

IPC: G06F40/40

Abstract: A method of training a deep learning model, a method of processing a natural language, an electronic device, and a storage medium are provided, which relate to a field of artificial intelligence, in particular to deep learning technology and natural language processing technology. The method includes: inputting first sample data into a first deep learning model to obtain a first output result; training the first deep learning model according to the first output result and a first target output result, the first target output result is obtained by processing the first sample data using a reference deep learning model; inputting second sample data into a second deep learning model to obtain a second output result; and training the second deep learning model according to the second output result and a second target output result, to obtain a trained second deep learning model.