公开(公告)号：US20240265256A1

公开(公告)日：2024-08-08

申请号：US18617095

申请日：2024-03-26

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Wenyong HUANG ,  Zhenhe ZHANG ,  Yu Ting YEUNG

IPC: G06N3/08 ,  G06F40/279 ,  G06V10/764 ,  G06V10/82 ,  G10L15/16

CPC classification number: G06N3/08 ,  G06F40/279 ,  G06V10/764 ,  G06V10/82 ,  G10L15/16

Abstract: This application provides a model training method and a related device. The method in this application includes: obtaining a first data sequence and a perturbed first data sequence; processing the perturbed first data sequence by using a first to-be-trained model to obtain a first feature sequence, and processing the first data sequence by using a second to-be-trained model to obtain a second feature sequence; training the first to-be-trained model and the second to-be-trained model based on the first feature sequence and the second feature sequence to obtain a first target model and a second target model; and fine-tuning the first target model or the second target model to obtain a third target model, where the third target model is used to obtain a label of a data sequence.

公开(公告)号：US20230088915A1

公开(公告)日：2023-03-23

申请号：US17994068

申请日：2022-11-25

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Wenyong HUANG ,  Yu Ting YEUNG ,  Xiao CHEN

IPC: G06F40/205 ,  G06F40/279

Abstract: This application provides a method and apparatus for sequence processing, relates to the field of artificial intelligence, and specifically relates to the field of sequence data processing. The method includes: receiving an input sequence (S410); performing self-attention calculation on a first element in the input sequence by using an element included in M windows, to obtain a representation of the first element, where each window includes one element or a plurality of consecutive elements in the input sequence, there is an interval of at least one element between different windows, at least one of the M windows does not include the first element, and M is an integer greater than or equal to 1 (S420); and obtaining, based on the representation of the first element, an output sequence corresponding to the input sequence (S430).