公开(公告)号：US20230297372A1

公开(公告)日：2023-09-21

申请号：US18074990

申请日：2022-12-05

Applicant: Google LLC

Inventor： William Lacy ,  Gregory Michael Thorson ,  Christopher Aaron Clark ,  Norman Paul Jouppi ,  Thomas Norrie ,  Andrew Everett Phelps

IPC: G06F9/30 ,  G06F15/80 ,  G06F9/38 ,  G06F7/58 ,  G06F13/36 ,  G06F13/40 ,  G06F13/42 ,  G06F17/16

CPC classification number: G06F9/3001 ,  G06F7/588 ,  G06F9/30032 ,  G06F9/30036 ,  G06F9/30043 ,  G06F9/30098 ,  G06F9/3891 ,  G06F13/36 ,  G06F13/4068 ,  G06F13/4282 ,  G06F15/8053 ,  G06F15/8092 ,  G06F17/16 ,  G06F15/8046 ,  G06N3/063

Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

公开(公告)号：US11551138B2

公开(公告)日：2023-01-10

申请号：US16495815

申请日：2018-02-08

Applicant: GOOGLE LLC

Inventor： Ian Moray Mclaren ,  Norman Paul Jouppi ,  Clifford Hsiang Chao ,  Gregory Michael Thorson ,  Bjarke Hammersholt Roune

IPC: G06F15/173 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.

公开(公告)号：US20220172033A1

公开(公告)日：2022-06-02

申请号：US17520919

申请日：2021-11-08

Applicant: Google LLC

Inventor： Jonathan Ross ,  Gregory Michael Thorson

IPC: G06N3/063 ,  G06N5/04 ,  G06N3/08 ,  G06N3/04 ,  G06F15/80

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing a layer output for a convolutional neural network layer, the method comprising: receiving a plurality of activation inputs; forming a plurality of vector inputs from the plurality of activation inputs, each vector input comprising values from a distinct region within the multi-dimensional matrix; sending the plurality of vector inputs to one or more cells along a first dimension of the systolic array; generating a plurality of rotated kernel structures from each of the plurality of kernel; sending each kernel structure and each rotated kernel structure to one or more cells along a second dimension of the systolic array; causing the systolic array to generate an accumulated output based on the plurality of value inputs and the plurality of kernels; and generating the layer output from the accumulated output.

公开(公告)号：US11170291B2

公开(公告)日：2021-11-09

申请号：US16857808

申请日：2020-04-24

Applicant: Google LLC

Inventor： Jonathan Ross ,  Gregory Michael Thorson

IPC: G06N3/063 ,  G06N5/04 ,  G06N3/08 ,  G06N3/04 ,  G06F15/80

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing a layer output for a convolutional neural network layer, the method comprising: receiving a plurality of activation inputs; forming a plurality of vector inputs from the plurality of activation inputs, each vector input comprising values from a distinct region within the multi-dimensional matrix; sending the plurality of vector inputs to one or more cells along a first dimension of the systolic array; generating a plurality of rotated kernel structures from each of the plurality of kernel; sending each kernel structure and each rotated kernel structure to one or more cells along a second dimension of the systolic array; causing the systolic array to generate an accumulated output based on the plurality of value inputs and the plurality of kernels; and generating the layer output from the accumulated output.

公开(公告)号：US20210318983A1

公开(公告)日：2021-10-14

申请号：US17354947

申请日：2021-06-22

Applicant: Google LLC

Inventor： Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam

IPC: G06F15/80 ,  G06F9/30 ,  G06F17/16 ,  G06F9/38

Abstract: A vector reduction circuit configured to reduce an input vector of elements comprises a plurality of cells, wherein each of the plurality of cells other than a designated first cell that receives a designated first element of the input vector is configured to receive a particular element of the input vector, receive, from another of the one or more cells, a temporary reduction element, perform a reduction operation using the particular element and the temporary reduction element, and provide, as a new temporary reduction element, a result of performing the reduction operation using the particular element and the temporary reduction element. The vector reduction circuit also comprises an output circuit configured to provide, for output as a reduction of the input vector, a new temporary reduction element corresponding to a result of performing the reduction operation using a last element of the input vector.

公开(公告)号：US11061854B2

公开(公告)日：2021-07-13

申请号：US16918448

申请日：2020-07-01

Applicant: Google LLC

Inventor： Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam

IPC: G06F15/80 ,  G06F9/30 ,  G06F17/16 ,  G06F9/38 ,  G06N3/08

Abstract: A vector reduction circuit configured to reduce an input vector of elements comprises a plurality of cells, wherein each of the plurality of cells other than a designated first cell that receives a designated first element of the input vector is configured to receive a particular element of the input vector, receive, from another of the one or more cells, a temporary reduction element, perform a reduction operation using the particular element and the temporary reduction element, and provide, as a new temporary reduction element, a result of performing the reduction operation using the particular element and the temporary reduction element. The vector reduction circuit also comprises an output circuit configured to provide, for output as a reduction of the input vector, a new temporary reduction element corresponding to a result of performing the reduction operation using a last element of the input vector.

公开(公告)号：US11016764B2

公开(公告)日：2021-05-25

申请号：US16843015

申请日：2020-04-08

Applicant: Google LLC

Inventor： William Lacy ,  Gregory Michael Thorson ,  Christopher Aaron Clark ,  Norman Paul Jouppi ,  Thomas Norrie ,  Andrew Everett Phelps

IPC: G06F9/302 ,  G06F9/312 ,  G06F15/80 ,  G06F13/40 ,  G06F7/57 ,  G06N3/063 ,  G06N20/00 ,  G06F17/16 ,  G06F9/30 ,  G06F9/38 ,  G06F7/58 ,  G06F13/36 ,  G06F13/42

Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

公开(公告)号：US10922057B2

公开(公告)日：2021-02-16

申请号：US16579604

申请日：2019-09-23

Applicant: Google LLC

Inventor： Jonathan Ross ,  Robert David Nuckolls ,  Christopher Aaron Clark ,  Chester Li ,  Gregory Michael Thorson

IPC: G06F7/78 ,  G06F7/00 ,  G06F7/76 ,  G11C7/10 ,  G11C8/04 ,  G11C19/28 ,  G06N3/063 ,  G06N3/08

Abstract: A circuit for transposing a matrix comprising reversal circuitry configured, for each of one or more diagonals of the matrix, to receive elements of the matrix in a first vector and generate a second vector that includes the elements of the matrix in an order that is a reverse of an order of the elements of the matrix in the first vector, and rotation circuitry configured, for each of the one or more diagonals of the matrix, to determine a number of positions by which to rotate the elements of the matrix in the second vector, receive the second vector of elements of the matrix, and generate a third vector that includes the elements of the matrix in the second vector in an order that is a rotation of the elements of the matrix in the second vector by the determined number of positions.

公开(公告)号：US20200334198A1

公开(公告)日：2020-10-22

申请号：US16918448

申请日：2020-07-01

Applicant: Google LLC

Inventor： Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam

IPC: G06F15/80 ,  G06F9/30 ,  G06F17/16 ,  G06F9/38

Abstract: A vector reduction circuit configured to reduce an input vector of elements comprises a plurality of cells, wherein each of the plurality of cells other than a designated first cell that receives a designated first element of the input vector is configured to receive a particular element of the input vector, receive, from another of the one or more cells, a temporary reduction element, perform a reduction operation using the particular element and the temporary reduction element, and provide, as a new temporary reduction element, a result of performing the reduction operation using the particular element and the temporary reduction element. The vector reduction circuit also comprises an output circuit configured to provide, for output as a reduction of the input vector, a new temporary reduction element corresponding to a result of performing the reduction operation using a last element of the input vector.

公开(公告)号：US20200218981A1

公开(公告)日：2020-07-09

申请号：US16824411

申请日：2020-03-19

Applicant: Google LLC

Inventor： Jonathan Ross ,  Norman Paul Jouppi ,  Andrew Everett Phelps ,  Reginald Clifford Young ,  Thomas Norrie ,  Gregory Michael Thorson ,  Dan Luu

IPC: G06N3/08 ,  G06N5/04 ,  G06N3/063 ,  G06F15/80

Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.