公开(公告)号：US20210295137A1

公开(公告)日：2021-09-23

申请号：US16822640

申请日：2020-03-18

Applicant: Arm Limited

Inventor： Urmish Ajit Thakker ,  Shidhartha Das ,  Ganesh Suryanarayan Dasika

IPC: G06N3/063 ,  G06N3/04 ,  G06F12/06

Abstract: The present disclosure advantageously provides a hybrid memory artificial neural network hardware accelerator that includes a communication bus interface, a static memory, a non-refreshed dynamic memory, a controller and a computing engine. The static memory stores at least a portion of an ANN model. The ANN model includes an input layer, one or more hidden layers and an output layer, ANN basis weights, input data and output data. The non-refreshed dynamic memory is configured to store ANN custom weights for the input, hidden and output layers, and output data. For each layer or layer portion, the computing engine generates the ANN custom weights based on the ANN basis weights, stores the ANN custom weights in the non-refreshed dynamic memory, executes the layer or layer portion, based on inputs and the ANN custom weights, to generate layer output data, and stores the layer output data.

公开(公告)号：US11663814B2

公开(公告)日：2023-05-30

申请号：US16855681

申请日：2020-04-22

Applicant: Arm Limited

Inventor： Urmish Ajit Thakker ,  Jin Tao ,  Ganesh Suryanarayan Dasika ,  Jesse Garrett Beu

IPC: G06N3/08 ,  G06N3/082 ,  G06F17/18 ,  G06K9/62 ,  G06N3/04

CPC classification number: G06N3/082 ,  G06F17/18 ,  G06K9/6267 ,  G06N3/0472

Abstract: The present disclosure advantageously provides a system and a method for skipping recurrent neural network (RNN) state updates using a skip predictor. Sequential input data are received and divided into sequences of input data values, each input data value being associated with a different time step for a pre-trained RNN model. At each time step, the hidden state vector for a prior time step is received from the pre-trained RNN model, and a determination, based on the input data value and the hidden state vector for at least one prior time step, is made whether to provide or not provide the input data value associated with the time step to the pre-trained RNN model for processing. When the input data value is not provided, the pre-trained RNN model does not update its hidden state vector. Importantly, the skip predictor is trained without retraining the pre-trained RNN model.

公开(公告)号：US11507841B2

公开(公告)日：2022-11-22

申请号：US16786096

申请日：2020-02-10

Applicant: Arm Limited

Inventor： Urmish Ajit Thakker ,  Ganesh Suryanarayan Dasika

IPC: G06N3/063 ,  G06T1/20 ,  G06N3/08 ,  G06F40/40 ,  G06T1/60

Abstract: The present disclosure advantageously provides a hardware accelerator for a natural language processing application including a first memory, a second memory, and a computing engine (CE). The first memory is configured to store a configurable NLM and a set of NLM fixed weights. The second memory is configured to store an ANN model, a set of ANN weights, a set of NLM delta weights, input data and output data. The set of NLM delta weights may be smaller than the set of NLM fixed weights, and each NLM delta weight corresponds to an NLM fixed weight. The CE is configured to execute the NLM, based on the input data, the set of NLM fixed weights and the set of NLM delta weights, to generate intermediate output data, and execute the ANN model, based on the intermediate output data and the set of ANN weights, to generate the output data.

公开(公告)号：US20220405597A1

公开(公告)日：2022-12-22

申请号：US17349780

申请日：2021-06-16

Applicant: Arm Limited

Inventor： Urmish Ajit Thakker ,  Jesse Garrett Beu ,  Dibakar Gope ,  Mark John O'Connor

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62 ,  G06K9/46 ,  G06K9/00 ,  G10L15/02 ,  G10L15/16

Abstract: Example methods, apparatuses, and/or articles of manufacture are disclosed that may be implemented, in whole or in part, using one or more computing devices to adapt a computing device to classify physical features in a deployment environment. In a particular implementation, computing resources may be selectively de-allocated from at least one of one or more elements of a computing architecture based, at least in part, on assessed impacts to the one or more elements of the computing architecture.

公开(公告)号：US11468305B2

公开(公告)日：2022-10-11

申请号：US16822640

申请日：2020-03-18

Applicant: Arm Limited

Inventor： Urmish Ajit Thakker ,  Shidhartha Das ,  Ganesh Suryanarayan Dasika

IPC: G06N3/063 ,  G06F12/06 ,  G06N3/04

Abstract: The present disclosure advantageously provides a hybrid memory artificial neural network hardware accelerator that includes a communication bus interface, a static memory, a non-refreshed dynamic memory, a controller and a computing engine. The static memory stores at least a portion of an ANN model. The ANN model includes an input layer, one or more hidden layers and an output layer, ANN basis weights, input data and output data. The non-refreshed dynamic memory is configured to store ANN custom weights for the input, hidden and output layers, and output data. For each layer or layer portion, the computing engine generates the ANN custom weights based on the ANN basis weights, stores the ANN custom weights in the non-refreshed dynamic memory, executes the layer or layer portion, based on inputs and the ANN custom weights, to generate layer output data, and stores the layer output data.

公开(公告)号：US20210248008A1

公开(公告)日：2021-08-12

申请号：US16786096

申请日：2020-02-10

Applicant: Arm Limited

Inventor： Urmish Ajit Thakker ,  Ganesh Suryanarayan Dasika

IPC: G06F9/50 ,  G06N3/063 ,  G06N3/08 ,  G06F40/40

Abstract: The present disclosure advantageously provides a hardware accelerator for a natural language processing application including a first memory, a second memory, and a computing engine (CE). The first memory is configured to store a configurable NLM and a set of NLM fixed weights. The second memory is configured to store an ANN model, a set of ANN weights, a set of NLM delta weights, input data and output data. The set of NLM delta weights may be smaller than the set of NLM fixed weights, and each NLM delta weight corresponds to an NLM fixed weight. The CE is configured to execute the NLM, based on the input data, the set of NLM fixed weights and the set of NLM delta weights, to generate intermediate output data, and execute the ANN model, based on the intermediate output data and the set of ANN weights, to generate the output data.

公开(公告)号：US20210056422A1

公开(公告)日：2021-02-25

申请号：US16855681

申请日：2020-04-22

Applicant: Arm Limited

Inventor： Urmish Ajit Thakker ,  Jin Tao ,  Ganesh Suryanarayan Dasika ,  Jesse Garrett Beu

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62 ,  G06F17/18

Abstract: The present disclosure advantageously provides a system and a method for skipping recurrent neural network (RNN) state updates using a skip predictor. Sequential input data are received and divided into sequences of input data values, each input data value being associated with a different time step for a pre-trained RNN model. At each time step, the hidden state vector for a prior time step is received from the pre-trained RNN model, and a determination, based on the input data value and the hidden state vector for at least one prior time step, is made whether to provide or not provide the input data value associated with the time step to the pre-trained RNN model for processing. When the input data value is not provided, the pre-trained RNN model does not update its hidden state vector. Importantly, the skip predictor is trained without retraining the pre-trained RNN model.