公开(公告)号：US20200257754A1

公开(公告)日：2020-08-13

申请号：US16826075

申请日：2020-03-20

Applicant: Google LLC

Inventor： Andrew Everett Phelps ,  Norman Paul Jouppi

IPC: G06F17/16 ,  G06N3/02 ,  G06F7/487 ,  G06F7/483 ,  G06F9/30 ,  G06N3/04 ,  G06N3/063

Abstract: Methods, systems, and apparatus for performing a matrix multiplication using a hardware circuit are described. An example method begins by obtaining an input activation value and a weight input value in a first floating point format. The input activation value and the weight input value are multiplied to generate a product value in a second floating point format that has higher precision than the first floating point format. A partial sum value is obtained in a third floating point format that has a higher precision than the first floating point format. The partial sum value and the product value are combined to generate an updated partial sum value that has the third floating point format.

公开(公告)号：US20200233663A1

公开(公告)日：2020-07-23

申请号：US16843015

申请日：2020-04-08

Applicant: Google LLC

Inventor： William Lacy ,  Gregory Michael Thorson ,  Christopher Aaron Clark ,  Norman Paul Jouppi ,  Thomas Norrie ,  Andrew Everett Phelps

IPC: G06F9/30 ,  G06F15/80 ,  G06F9/38 ,  G06F7/58 ,  G06F13/36 ,  G06F13/40 ,  G06F13/42 ,  G06F17/16

Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

公开(公告)号：US20200226202A1

公开(公告)日：2020-07-16

申请号：US16830894

申请日：2020-03-26

Applicant: Google LLC

Inventor： Andrew Everett Phelps ,  Norman Paul Jouppi

IPC: G06F17/16 ,  G06F5/01 ,  G06F9/30 ,  G06F15/80

Abstract: Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. The matrix multiply unit may include cells arranged in columns of the systolic array. Two chains of weight shift registers per column of the systolic array are in the matrix multiply unit. Each weight shift register is connected to only one chain and each cell is connected to only one weight shift register. A weight matrix register per cell is configured to store a weight input received from a weight shift register. A multiply unit is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.

公开(公告)号：US20190258694A1

公开(公告)日：2019-08-22

申请号：US16283913

申请日：2019-02-25

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam ,  Jonathan Ross ,  Christopher Aaron Clark

IPC: G06F17/16 ,  G06F9/30 ,  G06F7/76

Abstract: A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.

公开(公告)号：US10228947B2

公开(公告)日：2019-03-12

申请号：US15844192

申请日：2017-12-15

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Andrew Everett Phelps

IPC: G06F9/302 ,  G06F17/16 ,  G06F9/355 ,  G06F9/30 ,  G06F9/32 ,  G06F9/345 ,  G06F9/38

Abstract: Methods, systems, and apparatus, including an apparatus for processing an instruction for accessing a N-dimensional tensor, the apparatus including multiple tensor index elements and multiple dimension multiplier elements, where each of the dimension multiplier elements has a corresponding tensor index element. The apparatus includes one or more processors configured to obtain an instruction to access a particular element of a N-dimensional tensor, where the N-dimensional tensor has multiple elements arranged across each of the N dimensions, and where N is an integer that is equal to or greater than one; determine, using one or more tensor index elements of the multiple tensor index elements and one or more dimension multiplier elements of the multiple dimension multiplier elements, an address of the particular element; and output data indicating the determined address for accessing the particular element of the N-dimensional tensor.

公开(公告)号：US10216705B2

公开(公告)日：2019-02-26

申请号：US15966275

申请日：2018-04-30

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam ,  Jonathan Ross ,  Christopher Aaron Clark

IPC: G06F17/16 ,  G06F7/76 ,  G06F9/30

Abstract: A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.

公开(公告)号：US09959247B1

公开(公告)日：2018-05-01

申请号：US15496418

申请日：2017-04-25

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam ,  Jonathan Ross ,  Christopher Aaron Clark

IPC: G06F7/52 ,  G06F17/16

CPC classification number: G06F17/16 ,  G06F7/76 ,  G06F9/30032 ,  G06F9/30036

Abstract: A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.

公开(公告)号：US20240370526A1

公开(公告)日：2024-11-07

申请号：US18638285

申请日：2024-04-17

Applicant: Google LLC

Inventor： Andrew Everett Phelps ,  Norman Paul Jouppi

IPC: G06F17/16 ,  G06F7/483 ,  G06F7/487 ,  G06F9/30 ,  G06N3/02 ,  G06N3/048 ,  G06N3/063

Abstract: Methods, systems, and apparatus for performing a matrix multiplication using a hardware circuit are described. An example method begins by obtaining an input activation value and a weight input value in a first floating point format. The input activation value and the weight input value are multiplied to generate a product value in a second floating point format that has higher precision than the first floating point format. A partial sum value is obtained in a third floating point format that has a higher precision than the first floating point format. The partial sum value and the product value are combined to generate an updated partial sum value that has the third floating point format.

公开(公告)号：US20240362298A1

公开(公告)日：2024-10-31

申请号：US18638469

申请日：2024-04-17

Applicant: Google LLC

Inventor： Andrew Everett Phelps ,  Norman Paul Jouppi

IPC: G06F17/16 ,  G06F5/01 ,  G06F9/30 ,  G06F15/80 ,  G06N3/04 ,  G06N3/08

CPC classification number: G06F17/16 ,  G06F5/015 ,  G06F9/30101 ,  G06F15/8046 ,  G06F9/30032 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. The matrix multiply unit may include cells arranged in columns of the systolic array. Two chains of weight shift registers per column of the systolic array are in the matrix multiply unit. Each weight shift register is connected to only one chain and each cell is connected to only one weight shift register. A weight matrix register per cell is configured to store a weight input received from a weight shift register. A multiply unit is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.

公开(公告)号：US20240303297A1

公开(公告)日：2024-09-12

申请号：US18444249

申请日：2024-02-16

Applicant: Google LLC

Inventor： Andrew Everett Phelps ,  Norman Paul Jouppi

IPC: G06F17/16 ,  G06F5/01 ,  G06F9/30 ,  G06F15/80 ,  G06N3/04 ,  G06N3/08

CPC classification number: G06F17/16 ,  G06F5/015 ,  G06F9/30101 ,  G06F15/8046 ,  G06F9/30032 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. Each cell of the matrix multiply includes: a weight matrix register configured to receive a weight input from either a transposed or a non-transposed weight shift register; a transposed weight shift register configured to receive a weight input from a horizontal direction to be stored in the weight matrix register; a non-transposed weight shift register configured to receive a weight input from a vertical direction to be stored in the weight matrix register; and a multiply unit that is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.