公开(公告)号：US20240428070A1

公开(公告)日：2024-12-26

申请号：US18809757

申请日：2024-08-20

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Pengfei LI ,  Liangyou LI ,  Meng ZHANG

IPC: G06N3/08 ,  G06N3/0455 ,  G06N3/047

Abstract: A method of model training is disclosed. The method includes: obtaining a second embedding vector input to a decoder in a pre-trained language model, where the second embedding vector corresponds to a second data sequence. The second data sequence includes first sub-data, a masked to-be-predicted data unit, and second sub-data. The first sub-data is located before the masked to-be-predicted data unit in the second data sequence, and the second sub-data is located after the masked to-be-predicted data unit in the second data sequence. The method further includes: obtaining a hidden state based on a first embedding vector by using an encoder in the pre-trained language model (PLM); and predicting the masked to-be-predicted data unit based on the first sub-data, the second sub-data, and the hidden state by using the decoder in the PLM and an output layer of the decoder.

公开(公告)号：US20230162723A1

公开(公告)日：2023-05-25

申请号：US18151186

申请日：2023-01-06

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Tong CUI ,  Jinghui XIAO ,  Liangyou LI

IPC: G10L15/06 ,  G10L15/22 ,  G10L15/16 ,  G10L15/20 ,  G06F40/40 ,  G06F40/30 ,  G06F40/279

CPC classification number: G10L15/063 ,  G10L15/22 ,  G10L15/16 ,  G10L15/20 ,  G06F40/40 ,  G06F40/30 ,  G06F40/279

Abstract: This application discloses example text data processing method. One example method includes obtaining a target text. The target text can then be processed based on a noise generation model to obtain a noisy text, where when the noise generation model is trained, training data of the noise generation model at least includes a first text and a second text, the first text is a correct text corresponding to speech data, and the second text is obtained by performing speech recognition on the speech data by using a first speech recognition model. A text processing model can then be trained, by using at least the noisy text as training data, to obtain a trained text processing model.