公开(公告)号：US20240370701A1

公开(公告)日：2024-11-07

申请号：US18582349

申请日：2024-02-20

Applicant: Samsung Electronics Co., Ltd.

Inventor： Suhel Jaber ,  Julia Isabel White

IPC: G06N3/0455

Abstract: A method includes receiving an input by a self-attention machine learning model and generating a set of queries using the input. This method also includes generating at least one of two sets of keys using the input and two sets of values using the input. This method also includes determining an output of the self-attention machine learning model using the two sets of keys, the two sets of values, or both. Another method includes identifying a query position for the set of queries, identifying a key position for the two sets of keys, and when the query position is determined to be equal to the key position, calculating an attention score using a first set of the two sets of keys, or, when the query position is determined to be unequal to the key position, calculating the attention score using a second set of the two sets of keys.