公开(公告)号：US20240185047A1

公开(公告)日：2024-06-06

申请号：US18464935

申请日：2023-09-11

Applicant: Google LLC

Inventor： Jonathan Ross ,  Gregory Michael Thorson

IPC: G06N3/063 ,  G06F15/80 ,  G06N3/045 ,  G06N3/08 ,  G06N5/04

CPC classification number: G06N3/063 ,  G06F15/8046 ,  G06N3/045 ,  G06N3/08 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing a layer output for a convolutional neural network layer, the method comprising: receiving a plurality of activation inputs; forming a plurality of vector inputs from the plurality of activation inputs, each vector input comprising values from a distinct region within the multi-dimensional matrix; sending the plurality of vector inputs to one or more cells along a first dimension of the systolic array; generating a plurality of rotated kernel structures from each of the plurality of kernel; sending each kernel structure and each rotated kernel structure to one or more cells along a second dimension of the systolic array; causing the systolic array to generate an accumulated output based on the plurality of value inputs and the plurality of kernels; and generating the layer output from the accumulated output.

公开(公告)号：US11620513B2

公开(公告)日：2023-04-04

申请号：US16593321

申请日：2019-10-04

Applicant: Google LLC

Inventor： Jonathan Ross ,  Andrew Everett Phelps

IPC: G06N3/08 ,  G06N3/063 ,  G06N3/04 ,  G06N5/04 ,  G06N3/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing a layer output for a convolutional neural network layer, the method comprising: receiving the layer input, the layer input comprising a plurality of activation inputs, the plurality of activation inputs represented as a multi-dimensional matrix comprising a plurality of depth levels, each depth level being a respective matrix of distinct activation inputs from the plurality of activation inputs; sending each respective kernel matrix structure to a distinct cell along a first dimension of the systolic array; for each depth level, sending the respective matrix of distinct activation inputs to a distinct cell along a second dimension of the systolic array; causing the systolic array to generate an accumulated output from the respective matrices sent to the cells; and generating the layer output from the accumulated output.

公开(公告)号：US20220366255A1

公开(公告)日：2022-11-17

申请号：US17874573

申请日：2022-07-27

Applicant: Google LLC

Inventor： Jonathan Ross ,  Norman Paul Jouppi ,  Andrew Everett Phelps ,  Reginald Clifford Young ,  Thomas Norrie ,  Gregory Michael Thorson ,  Dan Luu

IPC: G06N3/08 ,  G06F15/80 ,  G06N3/063 ,  G06N5/04

Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.

公开(公告)号：US11281966B2

公开(公告)日：2022-03-22

申请号：US16053305

申请日：2018-08-02

Applicant: Google LLC

Inventor： Jonathan Ross

IPC: G06N3/063 ,  G06F15/80

Abstract: A circuit for performing neural network computations for a neural network, the circuit comprising: a systolic array comprising a plurality of cells; a weight fetcher unit configured to, for each of the plurality of neural network layers: send, for the neural network layer, a plurality of weight inputs to cells along a first dimension of the systolic array; and a plurality of weight sequencer units, each weight sequencer unit coupled to a distinct cell along the first dimension of the systolic array, the plurality of weight sequencer units configured to, for each of the plurality of neural network layers: shift, for the neural network layer, the plurality of weight inputs to cells along the second dimension of the systolic array over a plurality of clock cycles and where each cell is configured to compute a product of an activation input and a respective weight input using multiplication circuitry.

公开(公告)号：US20210125029A1

公开(公告)日：2021-04-29

申请号：US17060420

申请日：2020-10-01

Applicant: Google LLC

Inventor： Reginald Clifford Young ,  Jonathan Ross

IPC: G06N3/04 ,  G06N3/02 ,  G06N3/063 ,  G06F17/16

Abstract: Methods, systems, and apparatus for efficiently performing a computation of a convolutional neural network layer. One of the methods includes transforming a X by Y by Z input tensor into a X′ by Y′ by Z′ input tensor, wherein X′ is smaller than or equal to X, Y′ is smaller than or equal to Y, and Z′ is larger than or equal to Z; obtaining one or more modified weight matrices, wherein the modified weight matrices operate on the X′ by Y′ by Z′ input tensor to generate a U′ by V′ by W′ output tensor, and the U′ by V′ by W′ output tensor is a transformed U by V by W output tensor; and processing the X′ by Y′ by Z′ input tensor using the modified weight matrices to generate the U′ by V′ by W′ output tensor.

公开(公告)号：US10956537B2

公开(公告)日：2021-03-23

申请号：US16840972

申请日：2020-04-06

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam ,  Jonathan Ross ,  Christopher Aaron Clark

IPC: G06F17/16 ,  G06F7/76 ,  G06F9/30 ,  G06N3/063 ,  G06N3/08

Abstract: A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.

公开(公告)号：US20210019618A1

公开(公告)日：2021-01-21

申请号：US16915161

申请日：2020-06-29

Applicant: Google LLC

Inventor： Jonathan Ross ,  Norman Paul Jouppi ,  Andrew Everett Phelps ,  Reginald Clifford Young ,  Thomas Norrie ,  Gregory Michael Thorson ,  Dan Luu

IPC: G06N3/08 ,  G06F15/80 ,  G06N3/063 ,  G06N5/04

Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.

公开(公告)号：US10886923B1

公开(公告)日：2021-01-05

申请号：US16826020

申请日：2020-03-20

Applicant: Google LLC

Inventor： Jonathan Ross

IPC: H03K19/17732 ,  H03K19/17796 ,  H03K19/17736

Abstract: Methods, systems, and apparatus, including a system that includes a first integrated circuit chip configured to store application logic for one or more executable applications; and a second integrated circuit chip communicatively coupled to the first integrated circuit chip, the second integrated circuit chip including an instruction decoder configured to decode instructions for executing the one or more executable applications; and a communication interface configured to transmit the decoded instructions to the first integrated circuit chip to execute the one or more executable applications on the first integrated circuit chip.

公开(公告)号：US20200301996A1

公开(公告)日：2020-09-24

申请号：US16840972

申请日：2020-04-06

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam ,  Jonathan Ross ,  Christopher Aaron Clark

IPC: G06F17/16 ,  G06F7/76 ,  G06F9/30

Abstract: A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.

公开(公告)号：US20180253403A1

公开(公告)日：2018-09-06

申请号：US15966275

申请日：2018-04-30

Applicant: Google LLC

Inventor： Dong Hyuk Woo ,  Gregory Michael Thorson ,  Andrew Everett Phelps ,  Olivier Temam ,  Jonathan Ross ,  Christopher Aaron Clark

IPC: G06F17/16

CPC classification number: G06F17/16 ,  G06F7/76 ,  G06F9/30032 ,  G06F9/30036

Abstract: A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.