公开(公告)号：US10977338B1

公开(公告)日：2021-04-13

申请号：US16120386

申请日：2018-09-03

申请人： Perceive Corporation

发明人： Kenneth Duong ,  Jung Ko ,  Steven L. Teig

IPC分类号： G06F17/16 ,  G06T1/20 ,  G06N3/08 ,  G06N3/063 ,  G06N3/04 ,  G06K9/00

摘要： Some embodiments provide a method for executing a portion of a node of a machine-trained network. The method receives (i) multiple input values computed by previous nodes of the machine-trained network and (ii) for each of the input values, a corresponding predefined weight value. Each of the weight values is zero, a positive value, or a negation of the positive value. To compute a dot product of the input values with the weight values, the method passes to an adder circuit the input value for each input value with a corresponding positive weight value, the value zero for each input value with a corresponding weight value of zero, and a binary inversion of the input value for each input value with a corresponding negative weight value. After the adder circuit adds the values passed to it, the method adds an additional value based on the number of negative weight values.

公开(公告)号：US10937196B1

公开(公告)日：2021-03-02

申请号：US16246112

申请日：2019-01-11

申请人： Perceive Corporation

发明人： Ilyas Mohammed

IPC分类号： G06T9/00 ,  G06K9/62 ,  G06F17/16 ,  G06N3/08 ,  G06T3/40

摘要： Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.

公开(公告)号：US20210034982A1

公开(公告)日：2021-02-04

申请号：US16596177

申请日：2019-10-08

申请人： Perceive Corporation

发明人： Eric A. Sather ,  Steven L. Teig

IPC分类号： G06N3/08 ,  G06N3/04 ,  G06N3/063 ,  G06N5/04 ,  G06N20/00 ,  G06F7/483

摘要： Some embodiments of the invention provide a novel method for training a quantized machine-trained network. Some embodiments provide a method of scaling a feature map of a pre-trained floating-point neural network in order to match the range of output values provided by quantized activations in a quantized neural network. A quantization function is modified, in some embodiments, to be differentiable to fix the mismatch between the loss function computed in forward propagation and the loss gradient used in backward propagation. Variational information bottleneck, in some embodiments, is incorporated to train the network to be insensitive to multiplicative noise applied to each channel. In some embodiments, channels that finish training with large noise, for example, exceeding 100%, are pruned.

公开(公告)号：US10887537B1

公开(公告)日：2021-01-05

申请号：US16246142

申请日：2019-01-11

申请人： Perceive Corporation

发明人： Ilyas Mohammed

IPC分类号： H04N5/374 ,  G06T5/00 ,  G01S3/783 ,  H01L27/146 ,  G06T7/55 ,  H04N5/3745 ,  G02B26/08

摘要： Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.

公开(公告)号：US10740434B1

公开(公告)日：2020-08-11

申请号：US16120387

申请日：2018-09-03

申请人： Perceive Corporation

发明人： Kenneth Duong ,  Jung Ko ,  Steven L. Teig

IPC分类号： G06F17/16 ,  G06N3/08 ,  G06N3/063

摘要： Some embodiments provide an IC for implementing a machine-trained network with multiple layers. The IC includes a set of circuits to compute a dot product of (i) a first number of input values computed by other circuits of the IC and (ii) a set of predefined weight values, several of which are zero, with a weight value for each of the input values. The set of circuits includes (i) a dot product computation circuit to compute the dot product based on a second number of inputs and (ii) for each input value, at least two sets of wires for providing the input value to at least two of the dot product computation circuit inputs. The second number is less than the first number. Each input value with a corresponding weight value that is not equal to zero is provided to a different one of the dot product computation circuit inputs.

公开(公告)号：US20200250476A1

公开(公告)日：2020-08-06

申请号：US16852329

申请日：2020-04-17

申请人： Perceive Corporation

发明人： Eric A. Sather ,  Steven L. Teig ,  Andrew C. Mihal

IPC分类号： G06K9/62 ,  G06K9/00 ,  G06N3/08

摘要： Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

公开(公告)号：US12093816B1

公开(公告)日：2024-09-17

申请号：US16923002

申请日：2020-07-07

申请人： Perceive Corporation

发明人： Alexandru F. Drimbarean ,  Steven L. Teig

IPC分类号： G06N3/08 ,  G06N3/04

CPC分类号： G06N3/08 ,  G06N3/04

摘要： Some embodiments of the invention provide a method for configuring a network with multiple nodes. Each node generates an output value based on received input values and a set of weights that are previously trained to each have an initial value. For each weight, the method calculates a factor that represents a loss of accuracy to the network due to changing the weight from its initial value to a different value in a set of allowed values for the weight. Based on the factors, the method identifies a subset of the weights that have factors with values below a threshold. The method changes the values of each weight from its initial value to one of the values in its set of allowed values. The values of the identified subset are all changed to zero. The method trains the weights beginning with the changed values for each weight.

公开(公告)号：US11941533B1

公开(公告)日：2024-03-26

申请号：US16525469

申请日：2019-07-29

申请人： Perceive Corporation

发明人： Brian Thomas ,  Steven L. Teig

IPC分类号： G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06N3/10

CPC分类号： G06N3/105 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08

摘要： Some embodiments provide a compiler for optimizing the implementation of a machine-trained network (e.g., a neural network) on an integrated circuit (IC). The compiler of some embodiments receives a specification of a machine-trained network including multiple layers of computation nodes and generates a graph representing options for implementing the machine-trained network in the IC. The compiler, as part of generating the graph, in some embodiments, determines whether any set of channels contains no non-zero values (i.e., contains only zero values). For sets of channels that include no non-zero values, some embodiments perform a zero channel removal operation to remove all-zero channels wherever possible. In some embodiments, zero channel removal operations include removing input channels, removing output channels, forward propagation, and backward propagation of channels and constants.

公开(公告)号：US11921561B2

公开(公告)日：2024-03-05

申请号：US17827625

申请日：2022-05-27

申请人： Perceive Corporation

发明人： Jung Ko ,  Kenneth Duong ,  Steven L. Teig

IPC分类号： G06F1/32 ,  G06F1/3287

CPC分类号： G06F1/3287

摘要： For a neural network inference circuit that executes a neural network including multiple computation nodes at multiple layers for which data is stored in a plurality of memory banks, some embodiments provide a method for dynamically putting memory banks into a sleep mode of operation to conserve power. The method tracks the accesses to individual memory banks and, if a certain number of clock cycles elapse with no access to a particular memory bank, sends a signal to the memory bank indicating that it should operate in a sleep mode. Circuit components involved in dynamic memory sleep, in some embodiments, include a core RAM pipeline, a core RAM sleep controller, a set of core RAM bank select decoders, and a set of core RAM memory bank wrappers.

公开(公告)号：US11900238B1

公开(公告)日：2024-02-13

申请号：US16780842

申请日：2020-02-03

申请人： Perceive Corporation

发明人： Steven L. Teig ,  Eric A. Sather

IPC分类号： G06N3/048 ,  G06N20/00 ,  G06N3/084 ,  G06F17/18 ,  G06N3/047

CPC分类号： G06N3/048 ,  G06F17/18 ,  G06N3/047 ,  G06N3/084 ,  G06N20/00

摘要： Some embodiments provide a method for reducing complexity of a machine-trained (MT) network that receives input data and computes output data for each input data. The MT network includes multiple computation nodes that (i) generate output values and (ii) use output values of other computation nodes as input values. During training of the MT network, the method introduces probabilistic noise to the output values of a set of the computation nodes. the method determines a subset of the computation nodes for which the introduction of the probabilistic noise to the output value does not affect the computed output data for the network. The method removes the subset of computation nodes from the trained MT network.