公开(公告)号：US12073216B1

公开(公告)日：2024-08-27

申请号：US18109583

申请日：2023-02-14

Applicant: Google LLC

Inventor： Matthew Leever Hedlund ,  Christopher Aaron Clark ,  Andrew Everett Phelps ,  Thomas James Norrie ,  Sushma Honnavara-Prasad ,  Vinayak Anand Gokhale ,  Pareesa Ameneh Golnari

IPC: G06F9/30 ,  G06F17/16

CPC classification number: G06F9/30036 ,  G06F9/30032 ,  G06F17/16

Abstract: In a system including vector registers storing right-hand side data and left-hand side data, first and second matrix staging registers, and a systolic array of processing cells for conducting matrix multiplication operations using the right-hand side data and left-hand side data, one or more processors load the right-hand side data from the vector registers to the first matrix staging register based on an instruction indicating whether to transpose the right-hand side data, load the left-hand side data from the vector registers into the second matrix staging register based on another instruction indicating whether to transpose the left-hand side data, load the right-hand side data from the first matrix staging register into the systolic array, and, in a cycle of the matrix multiplication operation, pass one or more columns of the left-hand side data from the second matrix staging register to a column of the systolic array.

公开(公告)号：US20240220202A1

公开(公告)日：2024-07-04

申请号：US18168972

申请日：2023-02-14

Applicant: Google LLC

Inventor： Matthew Leever Hedlund ,  Christopher Aaron Clark ,  Andrew Everett Phelps ,  Thomas James Norrie ,  Norman Paul Jouppi ,  Sushma Honnavara-Prasad ,  Vinayak Anand Gokhale ,  Pareesa Ameneh Golnari

IPC: G06F7/544 ,  G06F7/485 ,  G06F7/487 ,  G06F15/80

CPC classification number: G06F7/5443 ,  G06F7/485 ,  G06F7/4876 ,  G06F15/8046

Abstract: A system and method for matrix multiplication using a systolic array configurable between multiple modes of operation. A systolic processor may receive a data type indicator for the matrix multiplication. For a first data type, the systolic processor may load the right-hand side data from the right-hand matrix register into the data processing cells of the systolic array between row 0 and row M−1, and pass the respective row of the left-hand side data through a corresponding row of the systolic array between rows 0 and M−1. For a second data type, the systolic processor may split each element of the left-hand side data and the right-hand side data into respective first and second element halves, and move each element half through a corresponding row of the systolic array between rows 0 and 2M−1.

公开(公告)号：US20240211534A1

公开(公告)日：2024-06-27

申请号：US18241805

申请日：2023-09-01

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam ,  Jonathan Ross ,  Christopher Aaron Clark

IPC: G06F17/16 ,  G06F7/76 ,  G06F9/30 ,  G06N3/063 ,  G06N3/084

CPC classification number: G06F17/16 ,  G06F7/76 ,  G06F9/30032 ,  G06F9/30036 ,  G06N3/063 ,  G06N3/084

Abstract: A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.

公开(公告)号：US20240168914A1

公开(公告)日：2024-05-23

申请号：US18429142

申请日：2024-01-31

Applicant: Google LLC

Inventor： Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam

IPC: G06F15/80 ,  G06F9/30 ,  G06F9/38 ,  G06F17/16

CPC classification number: G06F15/8053 ,  G06F9/3001 ,  G06F9/30036 ,  G06F9/3877 ,  G06F9/3897 ,  G06F17/16 ,  G06N3/084

Abstract: A vector reduction circuit configured to reduce an input vector of elements comprises a plurality of cells, wherein each of the plurality of cells other than a designated first cell that receives a designated first element of the input vector is configured to receive a particular element of the input vector, receive, from another of the one or more cells, a temporary reduction element, perform a reduction operation using the particular element and the temporary reduction element, and provide, as a new temporary reduction element, a result of performing the reduction operation using the particular element and the temporary reduction element. The vector reduction circuit also comprises an output circuit configured to provide, for output as a reduction of the input vector, a new temporary reduction element corresponding to a result of performing the reduction operation using a last element of the input vector.

公开(公告)号：US11989259B2

公开(公告)日：2024-05-21

申请号：US17985069

申请日：2022-11-10

Applicant: Google LLC

Inventor： Andrew Everett Phelps ,  Norman Paul Jouppi

IPC: G06F17/16 ,  G06F5/01 ,  G06F9/30 ,  G06F15/80 ,  G06N3/04 ,  G06N3/08

CPC classification number: G06F17/16 ,  G06F5/015 ,  G06F9/30101 ,  G06F15/8046 ,  G06F9/30032 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. The matrix multiply unit may include cells arranged in columns of the systolic array. Two chains of weight shift registers per column of the systolic array are in the matrix multiply unit. Each weight shift register is connected to only one chain and each cell is connected to only one weight shift register. A weight matrix register per cell is configured to store a weight input received from a weight shift register. A multiply unit is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.

公开(公告)号：US20240160909A1

公开(公告)日：2024-05-16

申请号：US18423203

申请日：2024-01-25

Applicant: Google LLC

Inventor： Thomas Norrie ,  Andrew Everett Phelps ,  Norman Paul Jouppi ,  Matthew Leever Hedlund

IPC: G06N3/063 ,  G06F9/54 ,  G06F13/16 ,  G06F15/80 ,  G06F17/16

CPC classification number: G06N3/063 ,  G06F9/544 ,  G06F13/16 ,  G06F15/8061 ,  G06F15/8092 ,  G06F17/16 ,  G06F2213/28

Abstract: Methods, systems, and apparatus, including computer-readable media, are described for a hardware circuit configured to implement a neural network. The circuit includes a first memory, respective first and second processor cores, and a shared memory. The first memory provides data for performing computations to generate an output for a neural network layer. Each of the first and second cores include a vector memory for storing vector values derived from the data provided by the first memory. The shared memory is disposed generally intermediate the first memory and at least one core and includes: i) a direct memory access (DMA) data path configured to route data between the shared memory and the respective vector memories of the first and second cores and ii) a load-store data path configured to route data between the shared memory and respective vector registers of the first and second cores.

公开(公告)号：US11922292B2

公开(公告)日：2024-03-05

申请号：US15931970

申请日：2020-05-14

Applicant: Google LLC

Inventor： Thomas Norrie ,  Andrew Everett Phelps ,  Norman Paul Jouppi ,  Matthew Leever Hedlund

IPC: G06N3/063 ,  G06F9/54 ,  G06F13/16 ,  G06F15/80 ,  G06F17/16

CPC classification number: G06N3/063 ,  G06F9/544 ,  G06F13/16 ,  G06F15/8061 ,  G06F15/8092 ,  G06F17/16 ,  G06F2213/28

Abstract: Methods, systems, and apparatus, including computer-readable media, are described for a hardware circuit configured to implement a neural network. The circuit includes a first memory, respective first and second processor cores, and a shared memory. The first memory provides data for performing computations to generate an output for a neural network layer. Each of the first and second cores include a vector memory for storing vector values derived from the data provided by the first memory. The shared memory is disposed generally intermediate the first memory and at least one core and includes: i) a direct memory access (DMA) data path configured to route data between the shared memory and the respective vector memories of the first and second cores and ii) a load-store data path configured to route data between the shared memory and respective vector registers of the first and second cores.

公开(公告)号：US11907330B2

公开(公告)日：2024-02-20

申请号：US18111468

申请日：2023-02-17

Applicant: Google LLC

Inventor： Andrew Everett Phelps ,  Norman Paul Jouppi

IPC: G06F17/16 ,  G06F15/80 ,  G06F5/01 ,  G06F9/30 ,  G06N3/04 ,  G06N3/08

CPC classification number: G06F17/16 ,  G06F5/015 ,  G06F9/30101 ,  G06F15/8046 ,  G06F9/30032 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. Each cell of the matrix multiply includes: a weight matrix register configured to receive a weight input from either a transposed or a non-transposed weight shift register; a transposed weight shift register configured to receive a weight input from a horizontal direction to be stored in the weight matrix register; a non-transposed weight shift register configured to receive a weight input from a vertical direction to be stored in the weight matrix register; and a multiply unit that is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.

公开(公告)号：US20230297372A1

公开(公告)日：2023-09-21

申请号：US18074990

申请日：2022-12-05

Applicant: Google LLC

Inventor： William Lacy ,  Gregory Michael Thorson ,  Christopher Aaron Clark ,  Norman Paul Jouppi ,  Thomas Norrie ,  Andrew Everett Phelps

IPC: G06F9/30 ,  G06F15/80 ,  G06F9/38 ,  G06F7/58 ,  G06F13/36 ,  G06F13/40 ,  G06F13/42 ,  G06F17/16

CPC classification number: G06F9/3001 ,  G06F7/588 ,  G06F9/30032 ,  G06F9/30036 ,  G06F9/30043 ,  G06F9/30098 ,  G06F9/3891 ,  G06F13/36 ,  G06F13/4068 ,  G06F13/4282 ,  G06F15/8053 ,  G06F15/8092 ,  G06F17/16 ,  G06F15/8046 ,  G06N3/063

Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

公开(公告)号：US11645223B2

公开(公告)日：2023-05-09

申请号：US17658764

申请日：2022-04-11

Applicant: Google LLC

Inventor： Kyle Nesbit ,  Andrew Everett Phelps

IPC: G06F15/173 ,  G06F3/06

CPC classification number: G06F15/17331 ,  G06F3/065 ,  G06F3/067 ,  G06F3/0611 ,  G06F3/0619 ,  G06F3/0643 ,  G06F3/0659

Abstract: A distributed storage system including memory hosts and at least one curator in communication with the memory hosts. Each memory host has memory, and the curator manages striping of data across the memory hosts. In response to a memory access request by a client in communication with the memory hosts and the curator, the curator provides the client a file descriptor mapping data stripes and data stripe replications of a file on the memory hosts for remote direct memory access of the file on the memory hosts.