公开(公告)号：US20250103341A1

公开(公告)日：2025-03-27

申请号：US18824034

申请日：2024-09-04

Applicant: Amazon Technologies, Inc.

Inventor： Kenneth Duong ,  Jung Ko ,  Steven L. Teig

IPC: G06F9/38 ,  G06F15/80 ,  G06N3/063 ,  G06N5/046

Abstract: Some embodiments provide a neural network inference circuit (NNIC) for executing a neural network that includes multiple computation nodes at multiple layers. The NNIC includes multiple core circuits including memories for storing input values for the computation nodes. The NNIC includes a set of post-processing circuits for computing output values of the computation nodes. The output values for a first layer are for storage in the core circuits as input values for a second layer. The NNIC includes an output bus that connects the post-processing circuits to the core circuits. The output bus is for (i) receiving a set of output values from the post-processing circuits, (ii) transporting the output values of the set to the core circuits based on configuration data specifying a core circuit at which each of the output values is to be stored, and (iii) aligning the output values for storage in the core circuits.

公开(公告)号：US12299068B2

公开(公告)日：2025-05-13

申请号：US18384529

申请日：2023-10-27

Applicant: Amazon Technologies, Inc.

Inventor： Kenneth Duong ,  Jung Ko ,  Steven L. Teig

IPC: G06F17/16 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06T1/20 ,  G06V40/16

Abstract: Some embodiments provide an IC for implementing a machine-trained network with multiple layers. The IC includes a set of circuits to compute a dot product of (i) a first number of input values computed by other circuits of the IC and (ii) a set of predefined weight values, several of which are zero, with a weight value for each of the input values. The set of circuits includes (i) a dot product computation circuit to compute the dot product based on a second number of inputs and (ii) for each input value, at least two sets of wires for providing the input value to at least two of the dot product computation circuit inputs. The second number is less than the first number. Each input value with a corresponding weight value that is not equal to zero is provided to a different one of the dot product computation circuit inputs.

公开(公告)号：US12265905B2

公开(公告)日：2025-04-01

申请号：US17984228

申请日：2022-11-09

Applicant: Amazon Technologies, Inc.

Inventor： Jung Ko ,  Kenneth Duong ,  Steven L. Teig

IPC: G06F7/544 ,  G06F1/03 ,  G06F5/01 ,  G06F9/30 ,  G06F17/10 ,  G06F17/16 ,  G06N3/048 ,  G06N3/06 ,  G06N3/063 ,  G06N3/08 ,  G06N3/084 ,  G06N5/04 ,  G06N5/046 ,  G06N20/00

Abstract: Some embodiments provide a method for a circuit that executes a neural network including multiple nodes. The method loads a set of weight values for a node into a set of weight value buffers, a first set of bits of each input value of a set of input values for the node into a first set of input value buffers, and a second set of bits of each of the input values into a second set of input value buffers. The method computes a first dot product of the weight values and the first set of bits of each input value and a second dot product of the weight values and the second set of bits of each input value. The method shifts the second dot product by a particular number of bits and adds the first dot product with the bit-shifted second dot product to compute a dot product for the node.