公开(公告)号：US10997492B2

公开(公告)日：2021-05-04

申请号：US15838273

申请日：2017-12-11

Applicant: NVIDIA Corporation

Inventor： Szymon Migacz ,  Hao Wu ,  Dilip Sequeira ,  Ujval Kapasi ,  Maxim Milakov ,  Slawomir Kierat ,  Zacky Zhou ,  Yilin Zhang ,  Alex Fit-Florea

IPC: G06N3/04 ,  G06N3/08 ,  G06N3/063 ,  G06N3/02 ,  G06N3/10 ,  G06N7/00 ,  G06T9/00

Abstract: Aspects of the present invention are directed to computer-implemented techniques for performing data compression and conversion between data formats of varying degrees of precision, and more particularly for improving the inferencing (application) of artificial neural networks using a reduced precision (e.g., INT8) data format. Embodiments of the present invention generate candidate conversions of data output, then employ a relative measure of quality to identify the candidate conversion with the greatest accuracy (i.e., least divergence from the original higher precision values). The representation can be then be used during inference to perform computations on the resulting output data.

公开(公告)号：US12299577B2

公开(公告)日：2025-05-13

申请号：US15624577

申请日：2017-06-15

Applicant: NVIDIA CORPORATION

Inventor： Boris Ginsburg ,  Sergei Nikolaev ,  Ahmad Kiswani ,  Hao Wu ,  Amir Gholaminejad ,  Slawomir Kierat ,  Michael Houston ,  Alex Fit-Florea

IPC: G06N3/084 ,  G06F17/16 ,  G06N3/04 ,  G06N3/045 ,  G06N3/088

Abstract: Aspects of the present invention are directed to computer-implemented techniques for improving the training of artificial neural networks using a reduced precision (e.g., float16) data format. Embodiments of the present invention rescale tensor values prior to performing matrix operations (such as matrix multiplication or matrix addition) to prevent overflow and underflow. To preserve accuracy throughout the performance of the matrix operations, the scale factors are defined using a novel data format to represent tensors, wherein a matrix is represented by the tuple X, where X=(a, v[.]), wherein a is a float scale factor and v[.] are scaled values stored in the float16 format. The value of any element X[i] according to this data format would be equal to a*v[i].

公开(公告)号：US20180211152A1

公开(公告)日：2018-07-26

申请号：US15838273

申请日：2017-12-11

Applicant: NVIDIA Corporation

Inventor： Szymon Migacz ,  Hao Wu ,  Dilip Sequeira ,  Ujval Kapasi ,  Maxim Milakov ,  Slawomir Kierat ,  Zacky Zhou ,  Yilin Zhang ,  Alex Fit-Florea

IPC: G06N3/04 ,  G06N3/08

CPC classification number: G06N3/04 ,  G06N3/0454 ,  G06N3/08 ,  G06N7/00

Abstract: Aspects of the present invention are directed to computer-implemented techniques for performing data compression and conversion between data formats of varying degrees of precision, and more particularly for improving the inferencing (application) of artificial neural networks using a reduced precision (e.g., INT8) data format. Embodiments of the present invention generate candidate conversions of data output, then employ a relative measure of quality to identify the candidate conversion with the greatest accuracy (i.e., least divergence from the original higher precision values). The representation can be then be used during inference to perform computations on the resulting output data.

公开(公告)号：US20240119267A1

公开(公告)日：2024-04-11

申请号：US17950009

申请日：2022-09-21

Applicant: NVIDIA Corporation

Inventor： Slawomir Kierat ,  Piotr Karpinski ,  Mateusz Sieniawski ,  Pawel Morkisz ,  Szymon Migacz ,  Linnan Wang ,  Chen-Han Yu ,  Satish Salian ,  Ashwath Aithal ,  Alexandru Fit-Florea

IPC: G06N3/04 ,  G06N3/08

CPC classification number: G06N3/0481 ,  G06N3/08

Abstract: Apparatuses, systems, and techniques to selectively use one or more neural network layers. In at least one embodiment, one or more neural network layers are selectively used based on, for example, one or more iteratively increasing neural network performance metrics.

公开(公告)号：US20210256348A1

公开(公告)日：2021-08-19

申请号：US17306171

申请日：2021-05-03

Applicant: NVIDIA Corporation

Inventor： Szymon Migacz ,  Hao Wu ,  Dilip Sequeira ,  Ujval Kapasi ,  Maxim Milakov ,  Slawomir Kierat ,  Zacky Zhou ,  Yilin Zhang ,  Alex Fit-Florea

IPC: G06N3/04 ,  G06N3/08

Abstract: Aspects of the present invention are directed to computer-implemented techniques for performing data compression and conversion between data formats of varying degrees of precision, and more particularly for improving the inferencing (application) of artificial neural networks using a reduced precision (e.g., INT8) data format. Embodiments of the present invention generate candidate conversions of data output, then employ a relative measure of quality to identify the candidate conversion with the greatest accuracy (i.e., least divergence from the original higher precision values). The representation can be then be used during inference to perform computations on the resulting output data.