公开(公告)号：US20250053811A1

公开(公告)日：2025-02-13

申请号：US18639937

申请日：2024-04-18

Applicant: Samsung Electronics Co., Ltd.

Inventor： Ahmed ELKORDY ,  Behnam BABAGHOLAMI MOHAMADABADI ,  Mostafa EL-KHAMY

IPC: G06N3/082 ,  G06N3/04

Abstract: A system and a method are disclosed for hardware-aware pruning of conformer networks. In some embodiments, the method includes: training a neural network, the training including: performing a first pruning operation, on the neural network, after a first training epoch, and performing a second pruning operation, on the neural network, after a second training epoch and after the first pruning operation, wherein each of the pruning operations results in a respective pruning fraction, the respective pruning fraction being a function of an index of a training epoch preceding the pruning operation.

公开(公告)号：US20220301296A1

公开(公告)日：2022-09-22

申请号：US17674832

申请日：2022-02-17

Applicant: Samsung Electronics Co., Ltd.

Inventor： Behnam BABAGHOLAMI MOHAMADABADI ,  Qingfeng LIU ,  Mostafa EL-KHAMY ,  Jungwon LEE

IPC: G06V10/82 ,  G06N5/04 ,  G06V10/778 ,  G06V10/776 ,  G06V10/774

Abstract: A system and a method to train a neural network are disclosed. A first image is weakly and strongly augmented. The first image, the weakly and strongly augmented first images are input into a feature extractor to obtain augmented features. Each weakly augmented first image is input to a corresponding first expert head to determine a supervised loss for each weakly augmented first image. Each strongly augmented first image is input to a corresponding second expert head to determine a diversity loss for each strongly augmented first image. The feature extractor is trained to minimize the supervised loss on weakly augmented first images and to minimize a multi-expert consensus loss on strongly augmented first images. Each first expert head is trained to minimize the supervised loss for each weakly augmented first image, and each second expert head is trained to minimize the diversity loss for each strongly augmented first image.