公开(公告)号：US20240403616A1

公开(公告)日：2024-12-05

申请号：US18500229

申请日：2023-11-02

Applicant: Intel Corporation

Inventor： Umer Iftikhar Cheema ,  Kevin Brady ,  Robert Simofi ,  Colm O Faolain ,  Deepak Abraham Mathaikutty ,  Arnab Raha ,  Dinakar Kondru ,  Gary Baugh ,  Darren Crews ,  Fergal Connor

IPC: G06N3/048

Abstract: An activation function in a neural network may be approximated by one or more linear functions. A linear function may correspond to a segment of the input range of the activation function, e.g., a linear segment. A programmable look-up table may store slopes and intercepts of linear functions. A post processing engine (PPE) array executing the activation function may determine that an input data element of the activation function falls into the linear segment and compute an output of the linear function using the input data element. The output of the linear function may be used as the approximated output of the activation function. Alternatively, the PPE array may determine that the input data element is in a saturation segment and use a fixed value associated with the saturation segment as the approximated output of the activation function.

公开(公告)号：US20240111830A1

公开(公告)日：2024-04-04

申请号：US18534035

申请日：2023-12-08

Applicant: Intel Corporation

Inventor： Umer Iftikhar Cheema ,  Robert Simofi ,  Deepak Abraham Mathaikutty ,  Arnab Raha ,  Dinakar Kondru

IPC: G06F17/17 ,  G06F1/03

CPC classification number: G06F17/17 ,  G06F1/0307

Abstract: A non-linear activation function in a neural network may be approximated by one or more linear functions. The input range may be divided into input segments, each of which corresponds to a different exponent in the input range of the activation function and includes input data elements having the exponent. Target accuracies may be assigned to the identified exponents based on a statistics analysis of the input data elements. The target accuracy of an input segment will be used to determine one or more linear functions that approximate the activation function for the input segment. An error of an approximation of the activation function by a linear function for the input segment may be within the target accuracy. The parameters of the linear functions may be stored in a look-up table (LUT). During the execution of the DNN, the LUT may be used to execute the activation function.