公开(公告)号：US20240232598A1

公开(公告)日：2024-07-11

申请号：US18469272

申请日：2023-09-18

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/045 ,  G06N3/086 ,  G06N3/10

CPC classification number: G06N3/063 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/045 ,  G06N3/086 ,  G06N3/10

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US11500959B2

公开(公告)日：2022-11-15

申请号：US16543282

申请日：2019-08-16

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman

IPC: G06F17/16 ,  G06F9/30 ,  G06F9/38 ,  G06F9/455 ,  G06F9/50 ,  G06F13/28

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.

公开(公告)号：US20210049231A1

公开(公告)日：2021-02-18

申请号：US16543282

申请日：2019-08-16

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman

IPC: G06F17/16 ,  G06F9/38 ,  G06F9/50 ,  G06F9/455 ,  G06F13/28 ,  G06F9/30

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.

公开(公告)号：US10733016B1

公开(公告)日：2020-08-04

申请号：US16395697

申请日：2019-04-26

Applicant: Google LLC

Inventor： Yuanzhong Xu ,  James M. Stichnoth ,  David Alexander Majnemer

IPC: G06F9/38 ,  G06F9/30 ,  H04N5/225 ,  G05B19/418 ,  G05B23/02 ,  G06F9/48

Abstract: Methods, systems, and apparatus for scheduling first-in-first-out instructions are described. In one aspect, a method includes receiving data representing code of a program to be executed by a processing unit comprising hardware processors. For each of one or more of the hardware processors, an order of independent groups of first-in-first-out (FIFO) instructions for execution by the hardware processor is identified in the data representing the code of the program. For each independent group of FIFO instructions for execution by the hardware processor, a path length metric that represents how long it will take to reach an end of the program from the independent group of FIFO instructions is determined. A new order of the independent groups of FIFO instructions for execution by the hardware processor is generated based at least on the path length metric for each independent group of FIFO instructions for execution by the hardware processor.

公开(公告)号：US20250148357A1

公开(公告)日：2025-05-08

申请号：US18504016

申请日：2023-11-07

Applicant: Google LLC

Inventor： Aditya Binodkumar Agrawal ,  Blake Alan Hechtman ,  Matthew Leever Hedlund ,  David Alexander Majnemer ,  Marissa Karen Ikonomidis

IPC: G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for compresses a machine learning model having a plurality of parameters. In one aspect, one of the methods includes obtaining trained values of a set of parameters for at least a portion of a machine learning model; identifying one or more dense ranges for the trained values; determining a least number of bits required to represent each trained value within the one or more dense ranges; identifying a second format having a range that is smaller than a range of the first format; and generating a compressed version of the at least a portion of the machine learning model.

公开(公告)号：US20230418797A1

公开(公告)日：2023-12-28

申请号：US18341697

申请日：2023-06-26

Applicant: Google LLC

Inventor： Felix Ren-Chyan Chern ,  Blake Alan Hechtman ,  Andrew Thomas Davis ,  Ruiqi Guo ,  Sanjiv Kumar ,  David Alexander Majnemer

IPC: G06F16/22 ,  G06F16/28

CPC classification number: G06F16/2237 ,  G06F16/285

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a kNN computation using a hardware accelerator. One of the methods includes obtaining a set of one or more query vectors; obtaining a set of database vectors; and performing, on a hardware accelerator and for each query vector in the set, a search for the k most similar database vectors to the query vector, comprising: computing, by circuitry of the hardware accelerator and for each query vector, a respective similarity value between the query vector and each database vector; and for each query vector, identifying, by the hardware accelerator and for each bin, (i) an index of the most similar database vector within the bin and (ii) the respective similarity value for the most similar database vector within the bin.

公开(公告)号：US11763142B2

公开(公告)日：2023-09-19

申请号：US17902776

申请日：2022-09-02

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/06 ,  G06N3/10 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/086 ,  G06N3/045

CPC classification number: G06N3/063 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/045 ,  G06N3/086 ,  G06N3/10

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US11221879B2

公开(公告)日：2022-01-11

申请号：US16919968

申请日：2020-07-02

Applicant: Google LLC

Inventor： Yuanzhong Xu ,  James M. Stichnoth ,  David Alexander Majnemer

IPC: G06F9/38 ,  G06F9/30 ,  G06F9/46 ,  G06F9/44 ,  G06F9/00 ,  G06F9/302 ,  G06F9/48

Abstract: Methods, systems, and apparatus for scheduling first-in-first-out instructions are described. In one aspect, a method includes receiving data representing code of a program to be executed by a processing unit comprising hardware processors. For each of one or more of the hardware processors, an order of independent groups of first-in-first-out (FIFO) instructions for execution by the hardware processor is identified in the data representing the code of the program. For each independent group of FIFO instructions for execution by the hardware processor, a path length metric that represents how long it will take to reach an end of the program from the independent group of FIFO instructions is determined. A new order of the independent groups of FIFO instructions for execution by the hardware processor is generated based at least on the path length metric for each independent group of FIFO instructions for execution by the hardware processor.

公开(公告)号：US11449739B2

公开(公告)日：2022-09-20

申请号：US16548555

申请日：2019-08-22

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06N3/08 ,  G06N3/04 ,  G06N3/10 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US20220414441A1

公开(公告)日：2022-12-29

申请号：US17902776

申请日：2022-09-02

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06F17/16 ,  G06F17/15 ,  G06F30/20 ,  G06N3/04 ,  G06F30/27 ,  G06N3/08 ,  G06N3/10 ,  G06F30/367 ,  G06F30/18

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.