公开(公告)号：US20240160896A1

公开(公告)日：2024-05-16

申请号：US18335685

申请日：2023-06-15

Applicant: QUALCOMM Incorporated

Inventor： Shashanka VENKATARAMANAN ,  Amir GHODRATI ,  Amirhossein HABIBIAN

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for improved attention-based machine learning. A first attention propagation output is generated using a first transformer block of a plurality of transformer blocks, this generation including processing input data for the first transformer block using a first self-attention sub-block of the first transformer block. The first attention propagation output is propagated to a second transformer block of the plurality of transformer blocks. An output for the second transformer block is generated, this generation including generating output features for the second transformer block based on the first attention propagation output.