公开(公告)号：US20250148358A1

公开(公告)日：2025-05-08

申请号：US18504117

申请日：2023-11-07

Applicant: QUALCOMM Incorporated

Inventor： Zhuojin LI ,  Hsin-Pai CHENG ,  Hong CAI ,  Sweta PRIYADARSHI ,  Kartikeya BHARDWAJ ,  Viswanath GANAPATHY ,  Chirag Sureshbhai PATEL ,  Fatih Murat PORIKLI

IPC: G06N20/00

Abstract: A processor-implemented method for training-free architecture searching for a transformer model includes generating a set of transformer model candidates for a target device. Each transformer model candidate of the set of transformer model candidates is initialized with random weights. A set of data samples are randomly sampled to produce random data samples for inputting at each transformer model candidate. An attention confidence score is computed for each transformer model candidate based on the random data samples and the random weights. A transformer model candidate for the target device is selected based on the attention confidence score.