公开(公告)号：US20180300617A1

公开(公告)日：2018-10-18

申请号：US15953388

申请日：2018-04-13

申请人： Microsoft Technology Licensing, LLC

发明人： Chad Balling McBRIDE ,  Amol Ashok AMBARDEKAR ,  Kent D. CEDOLA ,  Boris BOBROV ,  George PETRE ,  Larry Marvin WALL

IPC分类号： G06N3/063 ,  G06F12/02 ,  G06F13/16

摘要： An exemplary artificial intelligence/machine learning hardware computing environment having an exemplary DNN module cooperating with one or more memory components can perform data sharing and distribution as well reuse of a buffer data to reduce the number of memory component read/writes thereby enhancing overall hardware performance and reducing power consumption. Illustratively, data from a cooperating memory component is read according to a selected operation of the exemplary hardware and written to corresponding other memory component for use by one or more processing elements (e.g., neurons). The data is read in such a manner to optimize the engagement of the one or more processing elements for each processing cycle as well as to reuse data previously stored in the one or more cooperating memory components. Operatively, the written data is copied to a shadow memory buffer prior to being consumed by the processing elements.

公开(公告)号：US20180300616A1

公开(公告)日：2018-10-18

申请号：US15953330

申请日：2018-04-13

申请人： Microsoft Technology Licensing, LLC

发明人： Amol Ashok AMBARDEKAR ,  Boris BOBROV ,  Chad Balling McBRIDE ,  George PETRE ,  Kent D. CEDOLA ,  Larry Marvin WALL

IPC分类号： G06N3/063 ,  G06N3/04 ,  G06N3/08

摘要： A deep neural network (DNN) module is disclosed that can dynamically partition neuron workload to reduce power consumption. The DNN module includes neurons and a group partitioner and scheduler unit. The group partitioner and scheduler unit divides a workload for the neurons into partitions in order to maximize the number of neurons that can simultaneously process the workload. The group partitioner and scheduler unit then assigns a group of neurons to each of the partitions. The groups of neurons in the DNN module process the workload in their assigned partition to generate a partial output value. The neurons in each group can then sum their partial output values to generate a final output value for the workload. The neurons can be powered down once the groups of neurons have completed processing their assigned workload to reduce power consumption.

公开(公告)号：US20210232904A1

公开(公告)日：2021-07-29

申请号：US17232074

申请日：2021-04-15

申请人： MICROSOFT TECHNOLOGY LICENSING, LLC

发明人： Amol Ashok AMBARDEKAR ,  Aleksandar TOMIC ,  Chad Balling McBRIDE ,  George PETRE ,  Kent D. CEDOLA ,  Larry Marvin Wall ,  Boris BOBROV

IPC分类号： G06N3/063 ,  G06N3/04 ,  G06F12/0862 ,  G06F9/46 ,  G06F1/324 ,  G06F3/06 ,  G06F9/38 ,  G06F12/08 ,  G06F12/10 ,  G06F15/80 ,  G06F17/15 ,  G06N3/06 ,  G06N3/08 ,  G06N3/10 ,  H03M7/30 ,  H04L12/715 ,  H04L29/08 ,  G06F9/30 ,  G06F13/16 ,  G06F1/3234 ,  G06F12/02 ,  G06F13/28

摘要： The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as memory data management of a NN/DNN. Using vector quantization of neuron weight values, the processing of data by neurons can be optimize the number of operations as well as memory utilization to enhance the overall performance of a NN/DNN. Operatively, one or more contiguous segments of weight values can be converted into one or more vectors of arbitrary length and each of the one or more vectors can be assigned an index. The generated indexes can be stored in an exemplary vector quantization lookup table and retrieved by exemplary fast weight lookup hardware at run time on the fly as part of an exemplary data processing function of the NN as part of an inline de-quantization operation to obtain needed one or more neuron weight values.

公开(公告)号：US20180300614A1

公开(公告)日：2018-10-18

申请号：US15951106

申请日：2018-04-11

申请人： Microsoft Technology Licensing, LLC

发明人： Amol Ashok AMBARDEKAR ,  Kent D. CEDOLA ,  Larry Marvin WALL ,  Boris BOBROV ,  George PETRE ,  Chad Balling McBRIDE

IPC分类号： G06N3/063 ,  G06F13/28 ,  G06F17/15 ,  G06F13/16

摘要： A deep neural network (DNN) processor is configured to execute descriptors in layer descriptor lists. The descriptors define instructions for performing a pass of a DNN by the DNN processor. Several types of descriptors can be utilized: memory-to-memory move (M2M) descriptors; operation descriptors; host communication descriptors; configuration descriptors; branch descriptors; and synchronization descriptors. A DMA engine uses M2M descriptors to perform multi-dimensional strided DMA operations. Operation descriptors define the type of operation to be performed by neurons in the DNN processor and the activation function to be used by the neurons. M2M descriptors are buffered separately from operation descriptors and can be executed at soon as possible, subject to explicitly set dependencies. As a result, latency can be reduced and, consequently, the neurons can complete their processing faster. The DNN module can then be powered down earlier than it otherwise would have, thereby saving power.

公开(公告)号：US20230071352A1

公开(公告)日：2023-03-09

申请号：US18054851

申请日：2022-11-11

申请人： MICROSOFT TECHNOLOGY LICENSING, LLC

发明人： Joseph Leon CORKERY ,  Benjamin Eliot LUNDELL ,  Larry Marvin WALL ,  Chad Balling McBRIDE ,  Amol Ashok AMBARDEKAR ,  George PETRE ,  Kent D. CEDOLA ,  Boris BOBROV

IPC分类号： H03M7/30 ,  G06N3/04 ,  G06N3/063 ,  G06F12/0862 ,  G06F9/46 ,  G06F1/324 ,  G06F3/06 ,  G06F9/38 ,  G06F12/08 ,  G06F12/10 ,  G06F15/80 ,  G06F17/15 ,  G06N3/06 ,  G06N3/08 ,  G06N3/10 ,  H04L45/02 ,  H04L67/02 ,  G06F9/30 ,  H04L67/1001 ,  G06F9/48 ,  G06F12/02 ,  G06F13/16 ,  G06F1/3234 ,  G06F13/28

摘要： A deep neural network (“DNN”) module can compress and decompress neuron-generated activation data to reduce the utilization of memory bus bandwidth. The compression unit can receive an uncompressed chunk of data generated by a neuron in the DNN module. The compression unit generates a mask portion and a data portion of a compressed output chunk. The mask portion encodes the presence and location of the zero and non-zero bytes in the uncompressed chunk of data. The data portion stores truncated non-zero bytes from the uncompressed chunk of data. A decompression unit can receive a compressed chunk of data from memory in the DNN processor or memory of an application host. The decompression unit decompresses the compressed chunk of data using the mask portion and the data portion. This can reduce memory bus utilization, allow a DNN module to complete processing operations more quickly, and reduce power consumption.

公开(公告)号：US20180300606A1

公开(公告)日：2018-10-18

申请号：US15953356

申请日：2018-04-13

申请人： Microsoft Technology Licensing, LLC

发明人： Joseph Leon CORKERY ,  Benjamin Eliot LUNDELL ,  Larry Marvin WALL ,  Chad Balling McBRIDE ,  Amol Ashok AMBARDEKAR ,  George PETRE ,  Kent D. CEDOLA ,  Boris BOBROV

IPC分类号： G06N3/04 ,  G06N3/063 ,  H03M7/30

摘要： A deep neural network (“DNN”) module can compress and decompress neuron-generated activation data to reduce the utilization of memory bus bandwidth. The compression unit can receive an uncompressed chunk of data generated by a neuron in the DNN module. The compression unit generates a mask portion and a data portion of a compressed output chunk. The mask portion encodes the presence and location of the zero and non-zero bytes in the uncompressed chunk of data. The data portion stores truncated non-zero bytes from the uncompressed chunk of data. A decompression unit can receive a compressed chunk of data from memory in the DNN processor or memory of an application host. The decompression unit decompresses the compressed chunk of data using the mask portion and the data portion. This can reduce memory bus utilization, allow a DNN module to complete processing operations more quickly, and reduce power consumption.

公开(公告)号：US20180300604A1

公开(公告)日：2018-10-18

申请号：US15950550

申请日：2018-04-11

申请人： Microsoft Technology Licensing, LLC

发明人： Chad Balling McBRIDE ,  Amol Ashok AMBARDEKAR ,  Kent D. CEDOLA ,  George PETRE ,  Larry Marvin WALL ,  Boris BOBROV

IPC分类号： G06N3/04 ,  G06N3/063

摘要： A deep neural network (DNN) processor is configured to execute layer descriptors in layer descriptor lists. The descriptors define instructions for performing a forward pass of a DNN by the DNN processor. The layer descriptors can also be utilized to manage the flow of descriptors through the DNN module. For example, layer descriptors can define dependencies upon other descriptors. Descriptors defining a dependency will not execute until the descriptors upon which they are dependent have completed. Layer descriptors can also define a “fence,” or barrier, function that can be used to prevent the processing of upstream layer descriptors until the processing of all downstream layer descriptors is complete. The fence bit guarantees that there are no other layer descriptors in the DNN processing pipeline before the layer descriptor that has the fence to be asserted is processed.

公开(公告)号：US20180300601A1

公开(公告)日：2018-10-18

申请号：US15719351

申请日：2017-09-28

申请人： Microsoft Technology Licensing, LLC

发明人： Kent D. CEDOLA ,  Chad Balling McBRIDE ,  Amol Ashok AMBARDEKAR ,  George PETRE ,  Larry Marvin WALL ,  Boris BOBROV

IPC分类号： G06N3/04 ,  G06F3/06 ,  G06N3/063

摘要： Optimized memory usage and management is crucial to the overall performance of a neural network (NN) or deep neural network (DNN) computing environment. Using various characteristics of the input data dimension, an apportionment sequence is calculated for the input data to be processed by the NN or DNN that optimizes the efficient use of the local and external memory components. The apportionment sequence can describe how to parcel the input data (and its associated processing parameters—e.g., processing weights) into one or more portions as well as how such portions of input data (and its associated processing parameters) are passed between the local memory, external memory, and processing unit components of the NN or DNN. Additionally, the apportionment sequence can include instructions to store generated output data in the local and/or external memory components so as to optimize the efficient use of the local and/or external memory components.

公开(公告)号：US20200233820A1

公开(公告)日：2020-07-23

申请号：US16843800

申请日：2020-04-08

申请人： Microsoft Technology Licensing, LLC

发明人： Chad Balling McBRIDE ,  Timothy Hume HEIL ,  Amol Ashok AMBARDEKAR ,  George PETRE ,  Kent D. CEDOLA ,  Larry Marvin WALL ,  Boris BOBROV

IPC分类号： G06F13/16 ,  G06N3/04 ,  G06N3/063 ,  G06F12/0862 ,  G06F9/46 ,  G06F1/324 ,  G06F3/06 ,  G06F9/38 ,  G06F12/08 ,  G06F12/10 ,  G06F15/80 ,  G06F17/15 ,  G06N3/06 ,  G06N3/08 ,  G06N3/10 ,  H03M7/30 ,  H04L12/715 ,  H04L29/08 ,  G06F1/3234 ,  G06F12/02 ,  G06F13/28

摘要： An exemplary computing environment having a DNN module can maintain one or more bandwidth throttling mechanisms. Illustratively, a first throttling mechanism can specify the number of cycles to wait between transactions on a cooperating fabric component (e.g., data bus). Illustratively, a second throttling mechanism can be a transaction count limiter that operatively sets a threshold of a number of transactions to be processed during a given transaction sequence and limits the number of transactions such as multiple transactions in flight to not exceed the set threshold. In an illustrative operation, in executing these two exemplary calculated throttling parameters, the average bandwidth usage and the peak bandwidth usage can be limited. Operatively, with this fabric bandwidth control, the processing units of the DNN are optimized to process data across each transaction cycle resulting in enhanced processing and lower power consumption.

公开(公告)号：US20180300607A1

公开(公告)日：2018-10-18

申请号：US15813952

申请日：2017-11-15

申请人： Microsoft Technology Licensing, LLC

发明人： George PETRE ,  Chad Balling McBRIDE ,  Amol Ashok AMBARDEKAR ,  Kent D. CEDOLA ,  Larry Marvin WALL ,  Boris BOBROV

IPC分类号： G06N3/04 ,  G06N3/06 ,  G06N3/10 ,  G06F9/38

摘要： The performance of a neural network (NN) and/or deep neural network (DNN) can be limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. By inserting a selected padding in the input data to align the input data in memory, data read/writes can be optimized for processing by the NN/DNN thereby enhancing the overall performance of a NN/DNN. Operatively, an operations controller/iterator can generate one or more instructions that inserts the selected padding into the data. The data padding can be calculated using various characteristics of the input data as well as the NN/DNN as well as characteristics of the cooperating memory components. Padding on the output data can be utilized to support the data alignment at the memory components and the cooperating processing units of the NN/DNN.