公开(公告)号：US20240046067A1

公开(公告)日：2024-02-08

申请号：US18380581

申请日：2023-10-16

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Junqiu WEI ,  Yi LIAO ,  Xin JIANG ,  Qun LIU ,  Li QIAN

IPC: G06N3/04

CPC classification number: G06N3/04

Abstract: A data processing method includes: obtaining a first embedding vector for indicating a known data unit and a position of the known data unit and a second embedding vector for indicating a position of a to-be-predicted data unit; processing the first embedding vector by using a target encoder, to obtain an output vector; and processing the output vector and the second embedding vector by using a target prediction network, to obtain a to-be-predicted data unit. According to the method, M pieces of additional position information do not need to be separately set as input of the target encoder, and a quantity of latent variables of intermediate output of the target encoder is also consistent with a quantity of input embedding vectors, thereby reducing a computation amount and memory consumption of the target encoder.

公开(公告)号：US20230048031A1

公开(公告)日：2023-02-16

申请号：US17964165

申请日：2022-10-12

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Yi LIAO ,  Bowen LI ,  Hao ZHENG ,  Xin JIANG ,  Qun LIU

IPC: G06N3/08 ,  G06F40/279 ,  G06F40/103

Abstract: Relating to the field of artificial intelligence, and specifically relating to the field of natural language processing, a data processing method includes and an apparatus performs: determining original text samples, where masking processing is not performed on the original text samples; and performing mask processing on the original text samples to obtain mask training samples, where the mask processing makes mask proportions of the mask training samples unfixed, and the mask training samples each are used to train a pretrained language model PLM. Training the PLM by using the mask training samples whose mask proportions are unfixed can enhance mode diversity of the training samples of the PLM. Therefore, features learned by the PLM are also diversified, a generalization capability of the PLM can be improved, and a natural language understanding capability of the PLM obtained through training can be improved.

公开(公告)号：US20230177410A1

公开(公告)日：2023-06-08

申请号：US18161620

申请日：2023-01-30

Applicant: Huawei Technologies Co., Ltd.

Inventor： Yi LIAO ,  Xin JIANG ,  Xiao CHEN ,  Li QIAN ,  Qun LIU

IPC: G06N20/20 ,  G06F9/54

CPC classification number: G06N20/20 ,  G06F9/54

Abstract: A model training method applied to the field of artificial intelligence is disclosed. The method includes: sending a first submodel to a first device, where the first submodel is obtained by compressing a to-be-trained model; receiving a first gradient sent by the first device, where the first gradient is obtained when the first device trains the first submodel; and performing model training on the to-be-trained model based on at least the first gradient, to obtain an updated to-be-trained model. In the method, a server compresses the to-be-trained model and delivers the to-be-trained model to a terminal device, so that the terminal device does not need to train a large model with a same scale as that of the server.