公开(公告)号：US12079711B2

公开(公告)日：2024-09-03

申请号：US17186598

申请日：2021-02-26

Applicant: Google LLC

Inventor： Uday Kumar Dasari ,  Olivier Temam ,  Ravi Narayanaswami ,  Dong Hyuk Woo

IPC: G06N3/063 ,  G06F7/50 ,  G06F13/16 ,  G06F13/40 ,  G06F15/78 ,  G06F17/16 ,  G06N3/04 ,  G06N3/0464 ,  G06N20/00 ,  G11C11/22 ,  G11C11/54 ,  H01L25/065 ,  H01L25/18

CPC classification number: G06N3/063 ,  G06F15/7896 ,  G06N3/04 ,  G06F7/50 ,  G06F13/1668 ,  G06F13/4027 ,  G06F17/16 ,  G06N3/0464 ,  G06N20/00 ,  G11C11/22 ,  G11C11/54 ,  H01L25/0652 ,  H01L25/0657 ,  H01L25/18 ,  H01L2225/06541 ,  H01L2225/06589

Abstract: Apparatus and methods for processing neural network models are provided. The apparatus can comprise a plurality of identical artificial intelligence processing dies. Each artificial intelligence processing die among the plurality of identical artificial intelligence processing dies can include at least one inter-die input block and at least one inter-die output block. Each artificial intelligence processing die among the plurality of identical artificial intelligence processing dies is communicatively coupled to another artificial intelligence processing die among the plurality of identical artificial intelligence processing dies by way of one or more communication paths from the at least one inter-die output block of the artificial intelligence processing die to the at least one inter-die input block of the artificial intelligence processing die. Each artificial intelligence processing die among the plurality of identical artificial intelligence processing dies corresponds to at least one layer of a neural network.

公开(公告)号：US12061968B2

公开(公告)日：2024-08-13

申请号：US17845291

申请日：2022-06-21

Applicant: Google LLC

Inventor： Ravi Narayanaswami ,  Dong Hyuk Woo ,  Olivier Temam ,  Harshit Khaitan

IPC: G06N3/04 ,  G06F13/28 ,  G06N3/045 ,  G06N3/063

CPC classification number: G06N3/04 ,  G06F13/28 ,  G06N3/045 ,  G06N3/063

Abstract: A computer-implemented method that includes receiving, by a processing unit, an instruction that specifies data values for performing a tensor computation. In response to receiving the instruction, the method may include, performing, by the processing unit, the tensor computation by executing a loop nest comprising a plurality of loops, wherein a structure of the loop nest is defined based on one or more of the data values of the instruction. The tensor computation can be at least a portion of a computation of a neural network layer. The data values specified by the instruction may comprise a value that specifies a type of the neural network layer, and the structure of the loop nest can be defined at least in part by the type of the neural network layer.

公开(公告)号：US20240231819A1

公开(公告)日：2024-07-11

申请号：US18505743

申请日：2023-11-09

Applicant: Google LLC

Inventor： Olivier Temam ,  Ravi Narayanaswami ,  Harshit Khaitan ,  Dong Hyuk Woo

IPC: G06F9/30 ,  G06F9/38 ,  G06F13/28 ,  G06N3/04 ,  G06N3/045 ,  G06N3/063

CPC classification number: G06F9/3001 ,  G06F9/30036 ,  G06F9/30065 ,  G06F9/3824 ,  G06F13/28 ,  G06N3/04 ,  G06N3/045 ,  G06N3/063

Abstract: A computing unit is disclosed, comprising a first memory bank for storing input activations and a second memory bank for storing parameters used in performing computations. The computing unit includes at least one cell comprising at least one multiply accumulate (“MAC”) operator that receives parameters from the second memory bank and performs computations. The computing unit further includes a first traversal unit that provides a control signal to the first memory bank to cause an input activation to be provided to a data bus accessible by the MAC operator. The computing unit performs one or more computations associated with at least one element of a data array, the one or more computations being performed by the MAC operator and comprising, in part, a multiply operation of the input activation received from the data bus and a parameter received from the second memory bank.

公开(公告)号：US20230418677A1

公开(公告)日：2023-12-28

申请号：US18036506

申请日：2020-12-21

Applicant: Google LLC

Inventor： Temitayo Fadelu ,  Ravi Narayanaswami ,  JiHong Min ,  Dongdong Li ,  Suyog Gupta ,  Jason Jong Kyu Park

IPC: G06F9/50 ,  G06F9/48

CPC classification number: G06F9/5055 ,  G06F9/4881

Abstract: The present disclosure describes a system and method for preempting a long-running process with a higher priority process in a machine learning system, such as a hardware accelerator. The machine learning hardware accelerator can be a multi-chip system including semiconductor chips that can be application-specific integrated circuits (ASIC) designed to perform machine learning operations. An ASIC is an integrated circuit (IC) that is customized for a particular use.

公开(公告)号：US11501144B2

公开(公告)日：2022-11-15

申请号：US16569607

申请日：2019-09-12

Applicant: Google LLC

Inventor： Olivier Temam ,  Harshit Khaitan ,  Ravi Narayanaswami ,  Dong Hyuk Woo

IPC: G06N3/063 ,  G06N3/04 ,  G06F13/00 ,  G06F9/38 ,  G06F17/16

Abstract: One embodiment of an accelerator includes a computing unit; a first memory bank for storing input activations and a second memory bank for storing parameters used in performing computations, the second memory bank configured to store a sufficient amount of the neural network parameters on the computing unit to allow for latency below a specified level with throughput above a specified level. The computing unit includes at least one cell comprising at least one multiply accumulate (“MAC”) operator that receives parameters from the second memory bank and performs computations. The computing unit further includes a first traversal unit that provides a control signal to the first memory bank to cause an input activation to be provided to a data bus accessible by the MAC operator. The computing unit performs computations associated with at least one element of a data array, the one or more computations performed by the MAC operator.

公开(公告)号：US20220318594A1

公开(公告)日：2022-10-06

申请号：US17845291

申请日：2022-06-21

Applicant: Google LLC

Inventor： Ravi Narayanaswami ,  Dong Hyuk Woo ,  Olivier Temam ,  Harshit Khaitan

IPC: G06N3/04 ,  G06N3/063 ,  G06F13/28

Abstract: A computer-implemented method that includes receiving, by a processing unit, an instruction that specifies data values for performing a tensor computation. In response to receiving the instruction, the method may include, performing, by the processing unit, the tensor computation by executing a loop nest comprising a plurality of loops, wherein a structure of the loop nest is defined based on one or more of the data values of the instruction. The tensor computation can be at least a portion of a computation of a neural network layer. The data values specified by the instruction may comprise a value that specifies a type of the neural network layer, and the structure of the loop nest can be defined at least in part by the type of the neural network layer.

公开(公告)号：US10936942B2

公开(公告)日：2021-03-02

申请号：US15819753

申请日：2017-11-21

Applicant: Google LLC

Inventor： Uday Kumar Dasari ,  Olivier Temam ,  Ravi Narayanaswami ,  Dong Hyuk Woo

IPC: G06N3/063 ,  G06N3/04 ,  G06F15/78

Abstract: Apparatus and methods for processing neural network models are provided. The apparatus can comprise a plurality of identical artificial intelligence processing dies. Each artificial intelligence processing die among the plurality of identical artificial intelligence processing dies can include at least one inter-die input block and at least one inter-die output block. Each artificial intelligence processing die among the plurality of identical artificial intelligence processing dies is communicatively coupled to another artificial intelligence processing die among the plurality of identical artificial intelligence processing dies by way of one or more communication paths from the at least one inter-die output block of the artificial intelligence processing die to the at least one inter-die input block of the artificial intelligence processing die. Each artificial intelligence processing die among the plurality of identical artificial intelligence processing dies corresponds to at least one layer of a neural network.

公开(公告)号：US20210034697A1

公开(公告)日：2021-02-04

申请号：US16928242

申请日：2020-07-14

Applicant: Google LLC

Inventor： Ravi Narayanaswami ,  Rahul Nagarajan ,  Dong Hyuk Woo ,  Christopher Daniel Leary

IPC: G06F17/16 ,  G06F17/14

Abstract: Methods, systems, and apparatus, including a system for transforming sparse elements to a dense matrix. The system is configured to receive a request for an output matrix based on sparse elements including sparse elements associated with a first dense matrix and sparse elements associated with a second dense matrix; obtain the sparse elements associated with the first dense matrix fetched by a first group of sparse element access units; obtain the sparse elements associated with the second dense matrix fetched by a second group of sparse element access units; and transform the sparse elements associated with the first dense matrix and the sparse elements associated with the second dense matrix to generate the output dense matrix that includes the sparse elements associated with the first dense matrix and the sparse elements associated with the second dense matrix.

公开(公告)号：US10824692B2

公开(公告)日：2020-11-03

申请号：US16159450

申请日：2018-10-12

Applicant: Google LLC

Inventor： Anand Suresh Kane ,  Ravi Narayanaswami

IPC: G06F7/485 ,  G06F17/10 ,  G06F7/50 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a circuit configured to add multiple inputs. The circuit includes a first adder section that receives a first input and a second input and adds the inputs to generate a first sum. The circuit also includes a second adder section that receives the first and second inputs and adds the inputs to generate a second sum. An input processor of the circuit receives the first and second inputs, determines whether a relationship between the first and second inputs satisfies a set of conditions, and selects a high-power mode of the adder circuit or a low-power mode of the adder circuit using the determined relationship between the first and second inputs. The high-power mode is selected and the first and second inputs are routed to the second adder section when the relationship satisfies the set of conditions.

公开(公告)号：US10802956B2

公开(公告)日：2020-10-13

申请号：US16112307

申请日：2018-08-24

Applicant: Google LLC

Inventor： Olivier Temam ,  Harshit Khaitan ,  Ravi Narayanaswami ,  Dong Hyuk Woo

IPC: G06N5/00 ,  G06F12/02 ,  G06F12/1009 ,  G06N5/02 ,  G06N20/00 ,  G06F9/345 ,  G06F9/30

Abstract: Methods, systems, and apparatus, including an apparatus for accessing data. In some implementations, an apparatus includes address offset value elements that are each configured to store an address offset value. For each address offset value element, the apparatus can include address computation elements that each store a value used to determine the address offset value. One or more processors are configured to receive a program for performing computations using tensor elements of a tensor. The processor(s) can identify, in the program, a prologue or epilogue loop having a corresponding data array for storing values of the prologue or epilogue loop and populate, for a first address offset value element that corresponds to the prologue or epilogue loop, the address computation elements for the first address offset value element with respective values based at least on a number of iterations of the prologue or epilogue loop.