公开(公告)号：US20230117033A1

公开(公告)日：2023-04-20

申请号：US18084948

申请日：2022-12-20

Applicant: Samsung Electronics Co., Ltd.

Inventor： Junhaeng LEE ,  Seungwon LEE ,  Sangwon HA ,  Wonjo LEE

IPC: H03M7/24 ,  G06N3/063 ,  G06F7/483 ,  G06F7/499 ,  G06F7/544

Abstract: A method of generating a fixed-point quantized neural network includes analyzing a statistical distribution for each channel of floating-point parameter values of feature maps and a kernel for each channel from data of a pre-trained floating-point neural network, determining a fixed-point expression of each of the parameters for each channel statistically covering a distribution range of the floating-point parameter values based on the statistical distribution for each channel, determining fractional lengths of a bias and a weight for each channel among the parameters of the fixed-point expression for each channel based on a result of performing a convolution operation, and generating a fixed-point quantized neural network in which the bias and the weight for each channel have the determined fractional lengths.

公开(公告)号：US20240185029A1

公开(公告)日：2024-06-06

申请号：US18437370

申请日：2024-02-09

Applicant: Samsung Electronics Co., Ltd.

Inventor： Wonjo LEE ,  Seungwon LEE ,  Junhaeng LEE

IPC: G06N3/045 ,  G06N3/047 ,  G06N3/084

CPC classification number: G06N3/045 ,  G06N3/047 ,  G06N3/084

Abstract: According to a method and apparatus for neural network quantization, a quantized neural network is generated by performing learning of a neural network, obtaining weight differences between an initial weight and an updated weight determined by the learning of each cycle for each of layers in the first neural network, analyzing a statistic of the weight differences for each of the layers, determining one or more layers, from among the layers, to be quantized with a lower-bit precision based on the analyzed statistic, and generating a second neural network by quantizing the determined one or more layers with the lower-bit precision.

公开(公告)号：US20190122106A1

公开(公告)日：2019-04-25

申请号：US16106703

申请日：2018-08-21

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Junhaeng LEE ,  Hyunsun PARK ,  Yeongjae CHOI

IPC: G06N3/08 ,  G06N3/063 ,  G06F7/483

Abstract: A processor-implemented neural network method includes calculating individual update values for a weight assigned to a connection relationship between nodes included in a neural network; generating an accumulated update value by accumulating the individual update values in an accumulation buffer; and training the neural network by updating the weight using the accumulated update value in response to the accumulated update value being equal to or greater than a threshold value.

公开(公告)号：US20230206031A1

公开(公告)日：2023-06-29

申请号：US18116553

申请日：2023-03-02

Applicant: Samsung Electronics Co., Ltd.

Inventor： Wonjo LEE ,  Seungwon LEE ,  Junhaeng LEE

IPC: G06N3/045 ,  G06N3/084 ,  G06N3/047

CPC classification number: G06N3/045 ,  G06N3/084 ,  G06N3/047

Abstract: According to a method and apparatus for neural network quantization, a quantized neural network is generated by performing learning of a neural network, obtaining weight differences between an initial weight and an updated weight determined by the learning of each cycle for each of layers in the first neural network, analyzing a statistic of the weight differences for each of the layers, determining one or more layers, from among the layers, to be quantized with a lower-bit precision based on the analyzed statistic, and generating a second neural network by quantizing the determined one or more layers with the lower-bit precision.

公开(公告)号：US20200012936A1

公开(公告)日：2020-01-09

申请号：US16249279

申请日：2019-01-16

Applicant: Samsung Electronics Co., Ltd.

Inventor： Junhaeng LEE ,  Hyunsun PARK ,  Joonho SONG

IPC: G06N3/08 ,  G06N3/04

Abstract: A neural network method and apparatus are provided. A processor implemented neural network includes calculating respective individual gradient values for updating a weight of a neural network, calculating a residual gradient value based on an accumulated gradient value obtained by accumulating the individual gradient values and a bit digit representing the weight, tuning the respective individual gradient values to correspond to a bit digit of the residual gradient value, summing the tuned respective individual gradient values, the residual gradient value, and the weight, and updating the weight and the residual gradient value based on a result of the summing to train the neural network.

公开(公告)号：US20240112030A1

公开(公告)日：2024-04-04

申请号：US18529620

申请日：2023-12-05

Applicant: Samsung Electronics Co., Ltd.

Inventor： Junhaeng LEE ,  Hyunsun PARK ,  Sehwan LEE ,  Seungwon LEE

IPC: G06N3/08 ,  G06N3/0495

CPC classification number: G06N3/08 ,  G06N3/0495

Abstract: A neural network method and apparatus is provided. A processor-implemented neural network method includes a processor and a memory storing information, including stored predetermined precision parameters of a layer of a n neural network, about the layer, the method includes obtaining information about the layer in the memory indicative of the number of output classes; determining, based on the obtained information, a precision for the layer based on the number of output classes of the layer, wherein the precision is determined proportionally with respect to the obtained number of output classes; and processing new parameters, with a set precision, for the layer based on the stored parameter.

公开(公告)号：US20200026986A1

公开(公告)日：2020-01-23

申请号：US16282748

申请日：2019-02-22

Applicant: Samsung Electronics Co., Ltd.

Inventor： SangWon HA ,  Junhaeng LEE

IPC: G06N3/04 ,  G06N3/08

Abstract: A neural network method of parameter quantization includes obtaining channel profile information for first parameter values of a floating-point type in each channel included in each of feature maps based on an input in a first dataset to a floating-point parameters pre-trained neural network; determining a probability density function (PDF) type, for each channel, appropriate for the channel profile information based on a classification network receiving the channel profile information as a dataset; determining a fixed-point representation, based on the determined PDF type, for each channel, statistically covering a distribution range of the first parameter values; and generating a fixed-point quantized neural network based on the fixed-point representation determined for each channel.

公开(公告)号：US20180341857A1

公开(公告)日：2018-11-29

申请号：US15880690

申请日：2018-01-26

Applicant: Samsung Electronics Co., Ltd. ,  Seoul National University R&DB Foundation

Inventor： Junhaeng LEE ,  Sungjoo YOO ,  Eunhyeok PARK

IPC: G06N3/08 ,  G06N3/04

Abstract: Provided are a neural network method and an apparatus, the method including obtaining a set of floating point data processed in a layer included in a neural network, determining a weighted entropy based on data values included in the set of floating point data, adjusting quantization levels assigned to the data values based on the weighted entropy, and quantizing the data values included in the set of floating point data in accordance with the adjusted quantization levels.

公开(公告)号：US20230102087A1

公开(公告)日：2023-03-30

申请号：US17993740

申请日：2022-11-23

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Junhaeng LEE ,  Hyunsun PARK ,  Yeongjae CHOI

IPC: G06N3/08 ,  G06N3/063 ,  G06F7/483

Abstract: A processor-implemented neural network method includes calculating individual update values for a weight assigned to a connection relationship between nodes included in a neural network; generating an accumulated update value by adding the individual update values; and training the neural network by updating the weight using the accumulated update value in response to the accumulated update value being equal to or greater than a threshold value, wherein the threshold value is a value of 2n of an n-th bit of the weight, where the n-th bit is a bit of lesser significance than a bit in the weight representing a largest magnitude bit among all bits of the weight

公开(公告)号：US20200218962A1

公开(公告)日：2020-07-09

申请号：US16738338

申请日：2020-01-09

Applicant: Samsung Electronics Co., Ltd.

Inventor： Wonjo LEE ,  Seungwon LEE ,  Junhaeng LEE

IPC: G06N3/04 ,  G06N3/08

Abstract: According to a method and apparatus for neural network quantization, a quantized neural network is generated by performing learning of a neural network, obtaining weight differences between an initial weight and an updated weight determined by the learning of each cycle for each of layers in the first neural network, analyzing a statistic of the weight differences for each of the layers, determining one or more layers, from among the layers, to be quantized with a lower-bit precision based on the analyzed statistic, and generating a second neural network by quantizing the determined one or more layers with the lower-bit precision.