公开(公告)号：US20190377549A1

公开(公告)日：2019-12-12

申请号：US16001838

申请日：2018-06-06

Applicant: NVIDIA Corporation

Inventor： Jonah M. Alben ,  Paulius Micikevicius ,  Hao Wu ,  Ming Yiu Siu

IPC: G06F7/499

Abstract: A method, computer readable medium, and system are disclosed for rounding numerical values. A set of bits from an input value is identified as a rounding value. A second set of bits representing a second value is extracted from the input value and added with the rounding value to produce a sum. The sum is truncated to produce the rounded output value. Thus, the present invention provides a stochastic rounding technique that rounds up an input value as a function of a second value and a rounding value, both of which were obtained from the input value. When the second value and rounding value are obtained from consistent bit locations of the input value, the resulting output value is deterministic. Stochastic rounding, which is deterministic, is advantageously applicable in deep learning applications.

公开(公告)号：US20240078433A1

公开(公告)日：2024-03-07

申请号：US18385871

申请日：2023-10-31

Applicant: NVIDIA Corporation

Inventor： Jonah Alben ,  Paulius Micikevicius ,  Hao Wu

IPC: G06N3/084 ,  G06N3/04 ,  G06N3/063

CPC classification number: G06N3/084 ,  G06N3/04 ,  G06N3/063

Abstract: In training a deep neural network using reduced precision, gradient computation operates on larger values without affecting the rest of the training procedure. One technique trains the deep neural network to develop loss, scales the loss, computes gradients at a reduced precision, and reduces the magnitude of the computed gradients to compensate for scaling of the loss. In one example non-limiting arrangement, the training forward pass scales a loss value by some factor S and the weight update reduces the weight gradient contribution by 1/S. Several techniques can be used for selecting scaling factor S and adjusting the weight update.

公开(公告)号：US11842280B2

公开(公告)日：2023-12-12

申请号：US15971884

申请日：2018-05-04

Applicant: NVIDIA Corporation

Inventor： Jonah Alben ,  Paulius Micikevicius ,  Hao Wu

IPC: G06N3/084 ,  G06N3/04 ,  G06N3/063

CPC classification number: G06N3/084 ,  G06N3/04 ,  G06N3/063

Abstract: In training a deep neural network using reduced precision, gradient computation operates on larger values without affecting the rest of the training procedure. One technique trains the deep neural network to develop loss, scales the loss, computes gradients at a reduced precision, and reduces the magnitude of the computed gradients to compensate for scaling of the loss. In one example non-limiting arrangement, the training forward pass scales a loss value by some factor S and the weight update reduces the weight gradient contribution by 1/S. Several techniques can be used for selecting scaling factor S and adjusting the weight update.

公开(公告)号：US10684824B2

公开(公告)日：2020-06-16

申请号：US16001838

申请日：2018-06-06

Applicant: NVIDIA Corporation

Inventor： Jonah M. Alben ,  Paulius Micikevicius ,  Hao Wu ,  Ming Yiu Siu

IPC: G06F7/499

Abstract: A method, computer readable medium, and system are disclosed for rounding numerical values. A set of bits from an input value is identified as a rounding value. A second set of bits representing a second value is extracted from the input value and added with the rounding value to produce a sum. The sum is truncated to produce the rounded output value. Thus, the present invention provides a stochastic rounding technique that rounds up an input value as a function of a second value and a rounding value, both of which were obtained from the input value. When the second value and rounding value are obtained from consistent bit locations of the input value, the resulting output value is deterministic. Stochastic rounding, which is deterministic, is advantageously applicable in deep learning applications.

公开(公告)号：US20160350088A1

公开(公告)日：2016-12-01

申请号：US14723141

申请日：2015-05-27

Applicant: Nvidia Corporation

Inventor： Mahesh Ravishankar ,  Paulius Micikevicius ,  Vinod Grover

IPC: G06F9/45

CPC classification number: G06F8/4434 ,  G06F8/41 ,  G06F8/45

Abstract: A compiler and a method of compiling code that reduces memory bandwidth when processing code on a computer are provided herein. In one embodiment, the method includes: (1) automatically identifying a sequence of operations for fusing, wherein the sequence of operations correspond to instructions from a source code, (2) determining subdivisions of a final output of the sequence of operations, (3) determining input data and intermediate operations needed to obtain a final subdivision output for each of the subdivisions and (4) automatically generating code to fuse the sequence of operations employing the subdivisions, wherein the automatically identifying and the automatically generating are performed by a processor.

Abstract translation: 本文提供了一种编译器和编译在计算机上处​​理代码时减少内存带宽的代码的方法。 在一个实施例中，该方法包括：（1）自动识别用于融合的操作序列，其中所述操作序列对应于源代码的指令，（2）确定所述操作序列的最终输出的细分，（3 ）确定为每个子部分获得最终细分输出所需的输入数据和中间操作，以及（4）自动生成代码以融合采用该细分的操作序列，其中自动识别和自动生成由处理器执行。

公开(公告)号：US20220327101A1

公开(公告)日：2022-10-13

申请号：US17323490

申请日：2021-05-18

Applicant: NVIDIA Corporation

Inventor： Jeffrey Michael Pool ,  Chong Yu ,  Paulius Micikevicius

IPC: G06F16/215 ,  G06N3/04 ,  G06N3/08

Abstract: Apparatuses, systems, and techniques to transform data sets, such as matrices representing layers of neural networks, to increase sparsity and/or other characteristics of said data sets to improve performance in computations, such as neural network computations. In at least one embodiment, one or more subsets of data in one or more sets of data are rearranged as part of a process to increase sparsity in said one or more sets of data to satisfy one or more one or more structural sparsity constraints.

公开(公告)号：US10152310B2

公开(公告)日：2018-12-11

申请号：US14723141

申请日：2015-05-27

Applicant: Nvidia Corporation

Inventor： Mahesh Ravishankar ,  Paulius Micikevicius ,  Vinod Grover

IPC: G06F9/45 ,  G06F8/41

Abstract: A compiler and a method of compiling code that reduces memory bandwidth when processing code on a computer are provided herein. In one embodiment, the method includes: (1) automatically identifying a sequence of operations for fusing, wherein the sequence of operations correspond to instructions from a source code, (2) determining subdivisions of a final output of the sequence of operations, (3) determining input data and intermediate operations needed to obtain a final subdivision output for each of the subdivisions and (4) automatically generating code to fuse the sequence of operations employing the subdivisions, wherein the automatically identifying and the automatically generating are performed by a processor.