EFFICIENT SOFTMAX COMPUTATION WITH NO LOSS IN ACCURACY

    公开(公告)号:US20240320490A1

    公开(公告)日:2024-09-26

    申请号:US18734487

    申请日:2024-06-05

    申请人: Intel Corporation

    IPC分类号: G06N3/08 G06N3/048

    CPC分类号: G06N3/08 G06N3/048

    摘要: A modified 2-pass version of the SoftMax operation can be implemented to address reduce computational cost without loss of accuracy, in particular for deep learning neural networks such as transformer-based neural networks and large language models (LLMs). The first pass is modified to include two scalar operations at the end. At the end of the first pass, a first scalar operation is performed to calculate a logarithm of the denominator, and a second scalar operation is performed to calculate an operand value based on a sum of the logarithm of the denominator and the maximum value. The second pass is modified to perform addition and exponentiation. In the second pass, an element of an input tensor is subtracted by the operand value to obtain an exponent, and a base is raised to the exponent. The second pass avoids divisions.

    METHODS AND APPARATUS TO GENERATE OPTIMIZED MODELS FOR INTERNET OF THINGS DEVICES

    公开(公告)号:US20230011937A1

    公开(公告)日:2023-01-12

    申请号:US17869618

    申请日:2022-07-20

    申请人: Intel Corporation

    摘要: Example systems, methods, and apparatus to generate optimized models for Internet of Things device are disclosed. An example apparatus includes a data receiver to collect data from a sensor of an internet of things device based a first sampling frequency and a buffer having a first buffer size; a model trainer to train a model based on the data collected from the sensor; a buffer analyzer to select a second sampling frequency and to reduce the buffer to a second buffer size, the model trainer to update the model based on the second buffer size; and a platform analyzer to: determine a duration of time that that internet of things device will take to analyze sensor data based on the updated model.

    METHODS AND APPARATUS TO GENERATE OPTIMIZED MODELS FOR INTERNET OF THINGS DEVICES

    公开(公告)号:US20190140911A1

    公开(公告)日:2019-05-09

    申请号:US16236290

    申请日:2018-12-28

    申请人: Intel Corporation

    摘要: Example systems, methods, and apparatus to generate optimized models for Internet of Things device are disclosed. An example apparatus includes a data receiver to collect data from a sensor of an internet of things device based a first sampling frequency and a buffer having a first buffer size; a model trainer to train a model based on the data collected from the sensor; a buffer analyzer to select a second sampling frequency and to reduce the buffer to a second buffer size, the model trainer to update the model based on the second buffer size; and a platform analyzer to: determine a duration of time that that internet of things device will take to analyze sensor data based on the updated model.