公开(公告)号：US20230073835A1

公开(公告)日：2023-03-09

申请号：US17900126

申请日：2022-08-31

Applicant: Samsung Electronics Co., Ltd.

Inventor： Miao Yin ,  Burak Uzkent ,  Yilin Shen ,  Hongxia Jin

IPC: G06V10/70 ,  G06V10/774 ,  G06V10/776 ,  G06V10/74

Abstract: In one embodiment, a method includes accessing a batch B of a plurality of images, wherein each image in the batch is part of a training set of images used to train a vision transformer comprising a plurality of attention heads. The method further includes determining, for each attention head A, a similarity between (1) the output of the attention head evaluated using each image in the batch and the (2) output of each attention head evaluated using each image in the batch. The method further includes determining, based on the determined similarities, an importance score for each attention head; and pruning, based on the importance scores, one or more attention heads from the vision transformer.