DEVICE AND METHOD WITH TRANSFORMER MODEL IMPLEMENTATION

    公开(公告)号:EP4174723A1

    公开(公告)日:2023-05-03

    申请号:EP22204537.9

    申请日:2022-10-28

    摘要: A device and method with transformer model implementation are provided. The electronic device includes a processor configured to perform an inference by implementing a transformer model including a plurality of encoders and a plurality of decoders, and a memory configured to store instructions to be executed by the processor. Each of the encoders and the decoders includes an attention block that determines an attention value. The processor is configured to perform a first sub-softmax tile-wise operation in the attention block, perform a reduction operation to determine an adjustment factor based on a resulting value of the first sub-softmax operation, and perform a second sub-softmax tile-wise operation based on a resulting value of the reduction operation.