公开(公告)号：US20240152770A1

公开(公告)日：2024-05-09

申请号：US18411616

申请日：2024-01-12

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Hang XU ,  Xiaozhe REN ,  Yichun YIN ,  Li QIAN ,  Zhenguo LI ,  Xin JIANG ,  Jiahui GAO

IPC: G06N3/0985 ,  G06N3/04

CPC classification number: G06N3/0985 ,  G06N3/04

Abstract: This application relates to the artificial intelligence field, and discloses a neural network search method and a related apparatus. The neural network search method includes: constructing attention heads in transformer layers by sampling a plurality of candidate operators during model search, to construct a plurality of candidate neural networks, and comparing performance of the plurality of candidate neural networks to select a target neural network with higher performance. In this application, a transformer model is constructed with reference to model search, so that a new attention structure with better performance than an original self-attention mechanism can be generated, and effect in a wide range of downstream tasks is significantly improved.