公开(公告)号：US11461614B2

公开(公告)日：2022-10-04

申请号：US15838552

申请日：2017-12-12

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Daniel Ciubotariu ,  Mark Grobman ,  Alex Finkelstein

IPC: G06N3/04 ,  G06N3/063 ,  G06N20/00 ,  G06F17/10 ,  G06F5/01 ,  G06N3/08 ,  G06N3/02 ,  G06F12/02 ,  G06F12/06 ,  G06F30/30 ,  G06F30/27 ,  G06V10/40 ,  G06F7/501 ,  G06F7/523 ,  G06F9/50 ,  G06F13/16 ,  G06F9/30 ,  G06K9/62

Abstract: A novel and useful system and method of data driven quantization optimization of weights and input data in an artificial neural network (ANN). The system reduces quantization implications (i.e. error) in a limited resource system by employing the information available in the data actually observed by the system. Data counters in the layers of the network observe the data input thereto. The distribution of the data is used to determine an optimum quantization scheme to apply to the weights, input data, or both. The mechanism is sensitive to the data observed at the input layer of the network. As a result, the network auto-tunes to optimize the instance specific representation of the network. The network becomes customized (i.e. specialized) to the inputs it observes and better fits itself to the subset of the sample space that is applicable to its actual data flow. As a result, nominal process noise is reduced and detection accuracy improves. In addition, the mechanism enables the reduction of the representation space and further reduces the memory (and energy thereof) needed to represent the network properties.

公开(公告)号：US11263512B2

公开(公告)日：2022-03-01

申请号：US15943830

申请日：2018-04-03

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Hadar Zeitlin ,  Daniel Ciubotariu ,  Rami Feig

IPC: G06N3/04 ,  G06F12/02 ,  G06N3/063 ,  G06F30/27 ,  G06F7/501 ,  G06F7/523 ,  G06F9/50 ,  G06F17/10 ,  G06F5/01 ,  G06N3/08 ,  G06F13/16 ,  G06F9/30 ,  G06K9/46 ,  G06K9/62 ,  G06N3/02 ,  G06F12/06 ,  G06N20/00 ,  G06F30/30

Abstract: A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs) and incorporating strictly separate control and data planes. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.

公开(公告)号：US20180285719A1

公开(公告)日：2018-10-04

申请号：US15943830

申请日：2018-04-03

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Hadar Zeitlin ,  Daniel Ciubotariu ,  Rami Feig

IPC: G06N3/04 ,  G06N3/08 ,  G06F9/30

Abstract: A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs) and incorporating strictly separate control and data planes. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.

公开(公告)号：US11514291B2

公开(公告)日：2022-11-29

申请号：US15943976

申请日：2018-04-03

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Hadar Zeitlin ,  Daniel Ciubotariu ,  Rami Feig

IPC: G06N3/063 ,  G06N3/04 ,  G06F12/02 ,  G06F12/06 ,  G06N20/00 ,  G06F30/30 ,  G06F30/27 ,  G06V10/40 ,  G06F7/501 ,  G06F7/523 ,  G06F9/50 ,  G06F17/10 ,  G06F5/01 ,  G06N3/08 ,  G06F13/16 ,  G06F9/30 ,  G06K9/62 ,  G06N3/02

Abstract: A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs) and incorporating processing circuits having compute and local memory elements. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.

公开(公告)号：US20180285726A1

公开(公告)日：2018-10-04

申请号：US15943872

申请日：2018-04-03

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Hadar Zeitlin ,  Daniel Ciubotariu ,  Rami Feig

IPC: G06N3/063 ,  G06N3/04 ,  G06F9/50 ,  G06F17/10

Abstract: A novel and useful neural network (NN) processing core incorporating inter-device connectivity and adapted to implement artificial neural networks (ANNs). A chip-to-chip interface spreads a given ANN model across multiple devices in a seamless manner. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.

公开(公告)号：US11354563B2

公开(公告)日：2022-06-07

申请号：US15943845

申请日：2018-04-03

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Hadar Zeitlin ,  Daniel Ciubotariu ,  Rami Feig

IPC: G06F9/50 ,  G06N3/04 ,  G06F12/02 ,  G06N3/063 ,  G06F12/06 ,  G06N20/00 ,  G06F30/30 ,  G06F30/27 ,  G06V10/40 ,  G06F7/501 ,  G06F7/523 ,  G06F17/10 ,  G06F5/01 ,  G06N3/08 ,  G06F13/16 ,  G06F9/30 ,  G06K9/62 ,  G06N3/02

Abstract: A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs) and incorporating configurable and programmable sliding window based memory access. The memory mapping and allocation scheme trades off random and full access in favor of high parallelism and static mapping to a subset of the overall address space. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.

公开(公告)号：US20200005127A1

公开(公告)日：2020-01-02

申请号：US16569307

申请日：2019-09-12

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Daniel Ciubotariu

IPC: G06N3/063 ,  G06N3/04

Abstract: A novel and useful system and method of input alignment for streamlining vector operations that reduce the required memory read bandwidth. The input aligner as deployed in the NN processor, functions to facilitate the reuse of data read from memory and to avoid having to re-read that data in the context of neural network calculations. The input aligner functions to distribute input data (or weights) to the appropriate compute elements while consuming input data in a single cycle. Thus, the input aligner is operative to lower the required read bandwidth of layer input in an ANN. This reflects the fact that normally in practice, a vector multiplication is performed every time instance. This considers the fact that in many native calculations that take place in an ANN, the same data point is involved in multiple calculations.

公开(公告)号：US20180285718A1

公开(公告)日：2018-10-04

申请号：US15943800

申请日：2018-04-03

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Hadar Zeitlin ,  Daniel Ciubotariu ,  Rami Feig

IPC: G06N3/04 ,  G06N3/08

Abstract: A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs). The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.

公开(公告)号：US11675693B2

公开(公告)日：2023-06-13

申请号：US15943872

申请日：2018-04-03

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Hadar Zeitlin ,  Daniel Ciubotariu ,  Rami Feig

IPC: G06F12/02 ,  G06N3/063 ,  G06F12/06 ,  G06N20/00 ,  G06F30/30 ,  G06F30/27 ,  G06F18/00 ,  G06N3/045 ,  G06F7/501 ,  G06F7/523 ,  G06F9/50 ,  G06F17/10 ,  G06F5/01 ,  G06N3/08 ,  G06F13/16 ,  G06N3/04 ,  G06F9/30 ,  G06N3/084 ,  G06N3/02 ,  G06N3/082

CPC classification number: G06F12/0207 ,  G06F5/01 ,  G06F7/501 ,  G06F7/523 ,  G06F9/30054 ,  G06F9/5016 ,  G06F9/5027 ,  G06F12/02 ,  G06F12/0646 ,  G06F12/0692 ,  G06F13/1663 ,  G06F17/10 ,  G06F18/00 ,  G06F30/27 ,  G06F30/30 ,  G06N3/02 ,  G06N3/04 ,  G06N3/045 ,  G06N3/063 ,  G06N3/08 ,  G06N3/084 ,  G06N20/00 ,  G06N3/082 ,  Y02D10/00

Abstract: A novel and useful neural network (NN) processing core incorporating inter-device connectivity and adapted to implement artificial neural networks (ANNs). A chip-to-chip interface spreads a given ANN model across multiple devices in a seamless manner. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.

公开(公告)号：US11238334B2

公开(公告)日：2022-02-01

申请号：US16569307

申请日：2019-09-12

Applicant: Hailo Technologies Ltd.

Inventor： Avi Baum ,  Or Danon ,  Daniel Ciubotariu

IPC: G06N3/063 ,  G06F5/06 ,  G06F5/10 ,  G06F7/78 ,  G06F9/30 ,  G06N3/04 ,  G06N3/08

Abstract: A novel and useful system and method of input alignment for streamlining vector operations that reduce the required memory read bandwidth. The input aligner as deployed in the NN processor, functions to facilitate the reuse of data read from memory and to avoid having to re-read that data in the context of neural network calculations. The input aligner functions to distribute input data (or weights) to the appropriate compute elements while consuming input data in a single cycle. Thus, the input aligner is operative to lower the required read bandwidth of layer input in an ANN. This reflects the fact that normally in practice, a vector multiplication is performed every time instance. This considers the fact that in many native calculations that take place in an ANN, the same data point is involved in multiple calculations.