-
公开(公告)号：US11645535B2
公开(公告)日：2023-05-09
申请号：US16186468
申请日：2018-11-09
Applicant: Samsung Electronics Co., Ltd.
Inventor： Weiran Deng
CPC classification number: G06N3/084 , G06F5/01 , G06F7/5443
Abstract: A system and a method to normalize a deep neural network (DNN) in which a mean of activations of the DNN is set to be equal to about 0 for a training batch size of 8 or less, and a variance of the activations of the DNN is set to be equal to about a predetermined value for the training batch size. A minimization module minimizes a sum of a network loss of the DNN plus a sum of a product of a first Lagrange multiplier times the mean of the activations squared plus a sum of a product of a second Lagrange multiplier times a quantity of the variance of the activations minus one squared.
-
公开(公告)号：US20200234089A1
公开(公告)日：2020-07-23
申请号：US16844572
申请日：2020-04-09
Applicant: Samsung Electronics Co., Ltd.
Inventor： Georgios Georgiadis , Weiran Deng
Abstract: A system and method for pruning. A neural network includes a plurality of long short-term memory cells, each of which includes an input having a weight matrix Wc, an input gate having a weight matrix Wi, a forget gate having a weight matrix Wf, and an output gate having a weight matrix Wo. In some embodiments, after initial training, one or more of the weight matrices Wi, Wf, and Wo are pruned, and the weight matrix Wc is left unchanged. The neural network is then retrained, the pruned weights being constrained to remain zero during retraining.
-
公开(公告)号：US11475308B2
公开(公告)日：2022-10-18
申请号：US16396619
申请日：2019-04-26
Applicant: Samsung Electronics Co., Ltd.
Inventor： Georgios Georgiadis , Weiran Deng
Abstract: A system and a method generate a neural network that includes at least one layer having weights and output feature maps that have been jointly pruned and quantized. The weights of the layer are pruned using an analytic threshold function. Each weight remaining after pruning is quantized based on a weighted average of a quantization and dequantization of the weight for all quantization levels to form quantized weights for the layer. Output feature maps of the layer are generated based on the quantized weights of the layer. Each output feature map of the layer is quantized based on a weighted average of a quantization and dequantization of the output feature map for all quantization levels. Parameters of the analytic threshold function, the weighted average of all quantization levels of the weights and the weighted average of each output feature map of the layer are updated using a cost function.
-
公开(公告)号：US11461628B2
公开(公告)日：2022-10-04
申请号：US15864582
申请日：2018-01-08
Applicant: Samsung Electronics Co., Ltd.
Inventor： Weiran Deng
Abstract: A method includes: providing a deep neural networks (DNN) model comprising a plurality of layers, each layer of the plurality of layers includes a plurality of nodes; sampling a change of a weight for each of a plurality of weights based on a distribution function, each weight of the plurality of weights corresponds to each node of the plurality of nodes; updating the weight with the change of the weight multiplied by a sign of the weight; and training the DNN model by iterating the steps of sampling the change and updating the weight. The plurality of weights has a high rate of sparsity after the training.
-
公开(公告)号：US11449756B2
公开(公告)日：2022-09-20
申请号：US16186470
申请日：2018-11-09
Applicant: Samsung Electronics Co., Ltd.
Inventor： Weiran Deng
IPC: G06N3/08
Abstract: A system and method that provides balanced pruning of weights of a deep neural network (DNN) in which weights of the DNN are partitioned into a plurality of groups, a count of a number of non-zero weights is determined in each group, a variance of the count of weights in each group is determined, a loss function of the DNN is minimized using Lagrange multipliers with a constraint that the variance of the count of weights in each group is equal to 0, and the weights and the Lagrange multipliers are retrained by back-propagation.
-
Patent Agency Ranking
公开(公告)号：US20190138896A1
公开(公告)日：2019-05-09
申请号：US15864582
申请日：2018-01-08
Applicant: Samsung Electronics Co., Ltd.
Inventor： Weiran Deng
Abstract: A method includes: providing a deep neural networks (DNN) model comprising a plurality of layers, each layer of the plurality of layers includes a plurality of nodes; sampling a change of a weight for each of a plurality of weights based on a distribution function, each weight of the plurality of weights corresponds to each node of the plurality of nodes; updating the weight with the change of the weight multiplied by a sign of the weight; and training the DNN model by iterating the steps of sampling the change and updating the weight. The plurality of weights has a high rate of sparsity after the training.
-
公开(公告)号：US11250325B2
公开(公告)日：2022-02-15
申请号：US15894921
申请日：2018-02-12
Applicant: Samsung Electronics Co., Ltd.
Inventor： Weiran Deng , Georgios Georgiadis
Abstract: A technique to prune weights of a neural network using an analytic threshold function h(w) provides a neural network having weights that have been optimally pruned. The neural network includes a plurality of layers in which each layer includes a set of weights w associated with the layer that enhance a speed performance of the neural network, an accuracy of the neural network, or a combination thereof. Each set of weights is based on a cost function C that has been minimized by back-propagating an output of the neural network in response to input training data. The cost function C is also minimized based on a derivative of the cost function C with respect to a first parameter of the analytic threshold function h(w) and on a derivative of the cost function C with respect to a second parameter of the analytic threshold function h(w).
-
公开(公告)号：US10997272B2
公开(公告)日：2021-05-04
申请号：US16460564
申请日：2019-07-02
Applicant: Samsung Electronics Co., Ltd.
Inventor： Weiran Deng , Zhengping Ji
Abstract: A method of manufacturing an apparatus and a method of constructing an integrated circuit are provided. The method of manufacturing an apparatus includes forming the apparatus on a wafer or a package with at least one other apparatus, wherein the apparatus comprises a polynomial generator, a first matrix generator, a second matrix generator, a third matrix generator, and a convolution generator; and testing the apparatus, wherein testing the apparatus comprises testing the apparatus using one or more electrical to optical converters, one or more optical splitters that split an optical signal into two or more optical signals, and one or more optical to electrical converters.
-
公开(公告)号：US10657426B2
公开(公告)日：2020-05-19
申请号：US15937558
申请日：2018-03-27
Applicant: Samsung Electronics Co., Ltd.
Inventor： Georgios Georgiadis , Weiran Deng
Abstract: A system and method for pruning. A neural network includes a plurality of long short-term memory cells, each of which includes an input having a weight matrix Wc, an input gate having a weight matrix Wi, a forget gate having a weight matrix Wf, and an output gate having a weight matrix Wo. In some embodiments, after initial training, one or more of the weight matrices Wi, Wf, and Wo are pruned, and the weight matrix Wc is left unchanged. The neural network is then retrained, the pruned weights being constrained to remain zero during retraining.
-
公开(公告)号：US20190228274A1
公开(公告)日：2019-07-25
申请号：US15937558
申请日：2018-03-27
Applicant: Samsung Electronics Co., Ltd.
Inventor： Georgios Georgiadis , Weiran Deng
Abstract: A system and method for pruning. A neural network includes a plurality of long short-term memory cells, each of which includes an input having a weight matrix Wc, an input gate having a weight matrix Wi, a forget gate having a weight matrix Wf, and an output gate having a weight matrix Wo. In some embodiments, after initial training, one or more of the weight matrices Wi, Wf, and Wo are pruned, and the weight matrix Wc is left unchanged. The neural network is then retrained, the pruned weights being constrained to remain zero during retraining.
-
-
-
-
-
-
-
-
-
Search Results
15
records
(took 0.016 seconds)
