Invention Application
- Patent Title: APPARATUS AND METHOD WITH NEURAL NETWORK TRAINING BASED ON KNOWLEDGE DISTILLATION
-
Application No.: US17862779Application Date: 2022-07-12
-
Publication No.: US20230132630A1Publication Date: 2023-05-04
- Inventor: Eunhee KANG , Minsoo KANG , Bohyung HAN , Sehwan KI , HYONG EUK LEE
- Applicant: SAMSUNG ELECTRONICS CO., LTD. , Seoul National University R&DB Foundation
- Applicant Address: KR Suwon-si; KR Seoul
- Assignee: SAMSUNG ELECTRONICS CO., LTD.,Seoul National University R&DB Foundation
- Current Assignee: SAMSUNG ELECTRONICS CO., LTD.,Seoul National University R&DB Foundation
- Current Assignee Address: KR Suwon-si; KR Seoul
- Priority: KR10-2021-0148167 20211101,KR10-2022-0018390 20220211
- Main IPC: G06N3/04
- IPC: G06N3/04 ; G06N3/08

Abstract:
A method includes: generating, based on a student network result of an implemented student network provided with an input, a sample corresponding to a distribution of an energy-based model based on the student network result and a teacher network result of an implemented teacher network provided with the input; training model parameters of the energy-based model to decrease a value of the energy-based model, based on the teacher network result and the student network result; and training the implemented student network to increase the value of the energy-based model, based on the sample and the student network result.
Information query