Invention Publication
- Patent Title: NEURAL NETWORK DISTILLATION METHOD AND APPARATUS
-
Application No.: US18147297Application Date: 2022-12-28
-
Publication No.: US20230153615A1Publication Date: 2023-05-18
- Inventor: Yixing XU , Xinghao CHEN , Yunhe WANG , Chunjing XU
- Applicant: HUAWEI TECHNOLOGIES CO., LTD.
- Applicant Address: CN Shenzhen
- Assignee: HUAWEI TECHNOLOGIES CO., LTD.
- Current Assignee: HUAWEI TECHNOLOGIES CO., LTD.
- Current Assignee Address: CN Shenzhen
- Priority: CN 2010615537.4 2020.06.30
- Main IPC: G06N3/08
- IPC: G06N3/08 ; G06N3/045

Abstract:
The technology of this application relates to a neural network distillation method, applied to the field of artificial intelligence, and includes processing to-be-processed data by using a first neural network and a second neural network to obtain a first target output and a second target output, where the first target output is obtained by performing kernel function-based transformation on an output of the first neural network layer, and the second target output is obtained by performing kernel function-based transformation on an output of the second neural network layer. The method further includes performing knowledge distillation on the first neural network based on a target loss constructed by using the first target output and the second target output.
Information query