METHOD AND APPARATUS WITH TRANSFORMER MODEL TRAINING

    公开(公告)号:US20240135147A1

    公开(公告)日:2024-04-25

    申请号:US18450839

    申请日:2023-08-15

    CPC classification number: G06N3/0455

    Abstract: A device including processors configured to execute instructions and memories storing the instructions, which when executed by the processors configure the processors to perform an operation for training a transformer model having a plurality of encoders and a plurality of decoders by configuring the processors to identify the batches of training data into a plurality of micro-batches, select layer pairs for the plurality of micro-batches, assemble a processing order of the layer pairs, determining resource information to be allocated to the layer pairs, and allocate resources to the layer pairs based on the determined resource information to be allocated to the layer pairs, dependent con the processing order of the layer pairs.

    SEMICONDUCTOR DEVICE
    4.
    发明申请

    公开(公告)号:US20250113597A1

    公开(公告)日:2025-04-03

    申请号:US18978581

    申请日:2024-12-12

    Abstract: A semiconductor device includes a substrate including a first region, and a second region, a first gate structure and a second gate structure on the substrate of the first region, a third gate structure and a fourth gate structure on the substrate of the second region, a first interlayer insulating film on the substrate of the first region and including a first lower interlayer insulating film and a first upper interlayer insulating film, a second interlayer insulating film on the substrate of the second region and including a second lower interlayer insulating film and a second upper interlayer insulating film, a first contact between the first gate structure and the second gate structure and within the first interlayer insulating film, and a second contact formed between the third gate structure and the fourth gate structure and within the second interlayer insulating film.

    METHOD AND APPARATUS WITH TRANSFORMER MODEL TRAINING

    公开(公告)号:US20240232581A9

    公开(公告)日:2024-07-11

    申请号:US18450839

    申请日:2023-08-16

    CPC classification number: G06N3/0455

    Abstract: A device including processors configured to execute instructions and memories storing the instructions, which when executed by the processors configure the processors to perform an operation for training a transformer model having a plurality of encoders and a plurality of decoders by configuring the processors to identify the batches of training data into a plurality of micro-batches, select layer pairs for the plurality of micro-batches, assemble a processing order of the layer pairs, determining resource information to be allocated to the layer pairs, and allocate resources to the layer pairs based on the determined resource information to be allocated to the layer pairs, dependent con the processing order of the layer pairs.

    DEVICE AND METHOD WITH BATCH NORMALIZATION
    6.
    发明公开

    公开(公告)号:US20240184630A1

    公开(公告)日:2024-06-06

    申请号:US18526603

    申请日:2023-12-01

    CPC classification number: G06F9/5027 G06F5/01 G06F15/8046

    Abstract: A device and method with batch normalization are provided. An accelerator includes: core modules, each core module including a respective plurality of cores configured to perform a first convolution operation using feature map data and a weight; local reduction operation modules adjacent to the respective core modules, each including a respective plurality of local reduction operators configured to perform a first local operation that obtains first local statistical values of the corresponding core module; a global reduction operation module configured to perform a first global operation that generates first global statistical values of the core module based on the first local statistical values of the core modules; and a normalization operation module configured to perform a first normalization operation on the feature map data based on the first global statistical values.

    SEMICONDUCTOR DEVICE
    9.
    发明申请

    公开(公告)号:US20170141107A1

    公开(公告)日:2017-05-18

    申请号:US15333545

    申请日:2016-10-25

    Abstract: A semiconductor device includes a substrate including a first region, and a second region, a first gate structure and a second gate structure on the substrate of the first region, a third gate structure and a fourth gate structure on the substrate of the second region, a first interlayer insulating film on the substrate of the first region and including a first lower interlayer insulating film and a first upper interlayer insulating film, a second interlayer insulating film on the substrate of the second region and including a second lower interlayer insulating film and a second upper interlayer insulating film, a first contact between the first gate structure and the second gate structure and within the first interlayer insulating film, and a second contact formed between the third gate structure and the fourth gate structure and within the second interlayer insulating film.

Patent Agency Ranking