-
公开(公告)号：US20200167125A1
公开(公告)日：2020-05-28
申请号：US16200325
申请日：2018-11-26
Applicant: NVIDIA Corporation
Inventor： Alex Fit-Florea , Boris Ginsburg , Pooya Davoodi , Amir Gholaminejad
Abstract: A method, computer readable medium, and system are disclosed for rounding floating point values. Dynamic directional rounding is a rounding technique for floating point operations. A floating point operation (addition, subtraction, multiplication, etc.) is performed on an operand to compute a floating point result. A sign (positive or negative) of the operand is identified. In one embodiment, the sign determines a direction in which the floating point result is rounded (towards negative or positive infinity). When used for updating parameters of a neural network during backpropagation, dynamic directional rounding ensures that rounding is performed in the direction of the gradient.
-
公开(公告)号：US12299577B2
公开(公告)日：2025-05-13
申请号：US15624577
申请日：2017-06-15
Applicant: NVIDIA CORPORATION
Inventor： Boris Ginsburg , Sergei Nikolaev , Ahmad Kiswani , Hao Wu , Amir Gholaminejad , Slawomir Kierat , Michael Houston , Alex Fit-Florea
Abstract: Aspects of the present invention are directed to computer-implemented techniques for improving the training of artificial neural networks using a reduced precision (e.g., float16) data format. Embodiments of the present invention rescale tensor values prior to performing matrix operations (such as matrix multiplication or matrix addition) to prevent overflow and underflow. To preserve accuracy throughout the performance of the matrix operations, the scale factors are defined using a novel data format to represent tensors, wherein a matrix is represented by the tuple X, where X=(a, v[.]), wherein a is a float scale factor and v[.] are scaled values stored in the float16 format. The value of any element X[i] according to this data format would be equal to a*v[i].
-
公开(公告)号：US20210232366A1
公开(公告)日：2021-07-29
申请号：US17163855
申请日：2021-02-01
Applicant: NVIDIA Corporation
Inventor： Alex Fit-Florea , Boris Ginsburg , Pooya Davoodi , Amir Gholaminejad
Abstract: A method, computer readable medium, and system are disclosed for rounding floating point values. Dynamic directional rounding is a rounding technique for floating point operations. A floating point operation (addition, subtraction, multiplication, etc.) is performed on an operand to compute a floating point result. A sign (positive or negative) of the operand is identified. In one embodiment, the sign determines a direction in which the floating point result is rounded (towards negative or positive infinity). When used for updating parameters of a neural network during backpropagation, dynamic directional rounding ensures that rounding is performed in the direction of the gradient.
-
公开(公告)号：US10908878B2
公开(公告)日：2021-02-02
申请号：US16200325
申请日：2018-11-26
Applicant: NVIDIA Corporation
Inventor： Alex Fit-Florea , Boris Ginsburg , Pooya Davoodi , Amir Gholaminejad
Abstract: A method, computer readable medium, and system are disclosed for rounding floating point values. Dynamic directional rounding is a rounding technique for floating point operations. A floating point operation (addition, subtraction, multiplication, etc.) is performed on an operand to compute a floating point result. A sign (positive or negative) of the operand is identified. In one embodiment, the sign determines a direction in which the floating point result is rounded (towards negative or positive infinity). When used for updating parameters of a neural network during backpropagation, dynamic directional rounding ensures that rounding is performed in the direction of the gradient.
-
-
-
Search Results
4
records
(took 0.012 seconds)
