公开(公告)号：US20210248445A1

公开(公告)日：2021-08-12

申请号：US16840191

申请日：2020-04-03

Applicant: Google LLC

Inventor： Yanqi Zhou ,  Sudip Roy ,  Amirali Abdolrashidi ,  Daniel Lin-Kit Wong ,  Chao Ma ,  Qiumin Xu ,  Azalia Mirhoseini

IPC: G06N3/04 ,  G06K9/62

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing the execution of the operations of a neural network. One of the methods includes obtaining data representing a graph characterizing a plurality of operations of a neural network, wherein each node of the graph characterizes an operation of the neural network and each edge of the graph characterizes data dependency between the operations; processing the data representing the graph using a graph embedding neural network to generate an embedding of the graph; and processing the embedding of the graph using a policy neural network to generate a task output, wherein the task output comprises, for each of the plurality of operations of the neural network, a respective decision for a particular optimization task.

公开(公告)号：US10438113B2

公开(公告)日：2019-10-08

申请号：US16040186

申请日：2018-07-19

Applicant: Google LLC

Inventor： Benoit Steiner ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Hieu Hy Pham ,  Azalia Mirhoseini ,  Quoc V. Le

IPC: G06N3/04 ,  G06N20/00 ,  G06F16/901 ,  G06N3/063 ,  G06N3/08 ,  G06N5/04 ,  G06F9/50

Abstract: A method for determining a placement for machine learning model operations across multiple hardware devices includes receiving data specifying machine learning operations, and determining a placement that assigns each of the operations specified by the data to a respective device from the multiple hardware devices. Determining the placement includes: generating, from the data, a respective operation embedding for each of the operations; grouping the operations into multiple operation groups, comprising processing each of the respective operation embeddings using a grouper neural network having multiple grouper parameters, in which the grouper neural network is configured to, for each of the operations, process the operation embedding for the operation in accordance with first values of the grouper parameters to generate a grouper output that assigns the operation to an operation group from the multiple operation groups; and assigning each of the operation groups to a respective device from the multiple hardware devices.

公开(公告)号：US12086516B2

公开(公告)日：2024-09-10

申请号：US18310427

申请日：2023-05-01

Applicant: Google LLC

Inventor： Chian-min Richard Ho ,  William Hang ,  Mustafa Nazim Yazgan ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Azalia Mirhoseini ,  Emre Tuncer ,  Ya Wang ,  Anand Babu

IPC: G06F30/27 ,  G06F30/392

CPC classification number: G06F30/27 ,  G06F30/392

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US20240220768A1

公开(公告)日：2024-07-04

申请号：US18285977

申请日：2022-04-06

Applicant: Google LLC

Inventor： Dan Zhang ,  Safeen Huda ,  Azalia Mirhoseini ,  Anna Darling Goldie ,  Ebrahim Songhori

IPC: G06N3/04

CPC classification number: G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining a hardware datapath for a hardware accelerator computer chip.

公开(公告)号：US20240062062A1

公开(公告)日：2024-02-22

申请号：US18376362

申请日：2023-10-03

Applicant: Google LLC

Inventor： Samuel Bengio ,  Mohammad Norouzi ,  Benoit Steiner ,  Jeffrey Adgate Dean ,  Hieu Hy Pham ,  Azalia Mirhoseini ,  Quoc V. Le ,  Naveen Kumar ,  Yuefeng Zhou ,  Rasmus Munk Larsen

IPC: G06N3/08 ,  G06N5/04 ,  G06N3/10 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N5/04 ,  G06N3/105 ,  G06N3/044 ,  G06N3/045

Abstract: A method for determining a placement for machine learning model operations across multiple hardware devices is described. The method includes receiving data specifying a machine learning model to be placed for distributed processing on multiple hardware devices; generating, from the data, a sequence of operation embeddings, each operation embedding in the sequence characterizing respective operations necessary to perform the processing of the machine learning model; processing the sequence of operation embeddings using a placement recurrent neural network in accordance with first values of a plurality network parameters of the placement recurrent neural network to generate a network output that defines a placement of the operations characterized by the operation embeddings in the sequence across the plurality of devices; and scheduling the machine learning model for processing by the multiple hardware devices by placing the operations on the multiple devices according to the placement defined by the network output.

公开(公告)号：US11657289B2

公开(公告)日：2023-05-23

申请号：US16840191

申请日：2020-04-03

Applicant: Google LLC

Inventor： Yanqi Zhou ,  Sudip Roy ,  Amirali Abdolrashidi ,  Daniel Lin-Kit Wong ,  Chao Ma ,  Qiumin Xu ,  Azalia Mirhoseini

IPC: G06N3/04 ,  G06K9/62 ,  G06N3/049

CPC classification number: G06N3/0454 ,  G06K9/6231 ,  G06K9/6262 ,  G06K9/6296 ,  G06N3/049

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing the execution of the operations of a neural network. One of the methods includes obtaining data representing a graph characterizing a plurality of operations of a neural network, wherein each node of the graph characterizes an operation of the neural network and each edge of the graph characterizes data dependency between the operations; processing the data representing the graph using a graph embedding neural network to generate an embedding of the graph; and processing the embedding of the graph using a policy neural network to generate a task output, wherein the task output comprises, for each of the plurality of operations of the neural network, a respective decision for a particular optimization task.

公开(公告)号：US20210334445A1

公开(公告)日：2021-10-28

申请号：US17238128

申请日：2021-04-22

Applicant: Google LLC

Inventor： Anna Darling Goldie ,  Azalia Mirhoseini ,  Ebrahim Songhori ,  Wenjie Jiang ,  Shen Wang ,  Roger David Carpenter ,  Young-Joon Lee ,  Mustafa Nazim Yazgan ,  Chian-min Richard Ho ,  Quoc V. Le ,  James Laudon ,  Jeffrey Adgate Dean ,  Kavya Srinivasa Setty ,  Omkar Pathak

IPC: G06F30/392 ,  G06F30/398 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip placement. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip placement, comprising placing a respective macro node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the macro node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US20200175216A1

公开(公告)日：2020-06-04

申请号：US16703837

申请日：2019-12-04

Applicant: Google LLC

Inventor： Chian-min Richard Ho ,  William Hang ,  Mustafa Nazim Yazgan ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Azalia Mirhoseini ,  Emre Tuncer ,  Ya Wang ,  Anand Babu

IPC: G06F30/27 ,  G06F30/392

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US20250021799A1

公开(公告)日：2025-01-16

申请号：US18776868

申请日：2024-07-18

Applicant: Google LLC

Inventor： Noam M. Shazeer ,  Azalia Mirhoseini ,  Krzysztof Stanislaw Maziarz

IPC: G06N3/045 ,  G06N3/08

Abstract: A system includes a neural network that includes a Mixture of Experts (MoE) subnetwork between a first neural network layer and a second neural network layer. The MoE subnetwork includes multiple expert neural networks. Each expert neural network is configured to process a first layer output generated by the first neural network layer to generate a respective expert output. The MoE subnetwork further includes a gating subsystem that selects, based on the first layer output, one or more of the expert neural networks and determine a respective weight for each selected expert neural network, provides the first layer output as input to each of the selected expert neural networks, combines the expert outputs generated by the selected expert neural networks in accordance with the weights for the selected expert neural networks to generate an MoE output, and provides the MoE output as input to the second neural network layer.

公开(公告)号：US20240095424A1

公开(公告)日：2024-03-21

申请号：US17890370

申请日：2022-08-18

Applicant: Google LLC

Inventor： Ebrahim Mohammadgholi Songhori ,  Shen Wang ,  Azalia Mirhoseini ,  Anna Goldie ,  Roger Carpenter ,  Wenjie Jiang ,  Young-Joon Lee ,  James Laudon

IPC: G06F30/27 ,  G06F30/392

CPC classification number: G06F30/27 ,  G06F30/392

Abstract: Aspects of the disclosure are directed to automatically determining floor planning in chips, which factors in memory macro alignment. A deep reinforcement learning (RL) agent can be trained to determine optimal placements for the memory macros, where memory macro alignment can be included as a regularization cost to be added to the placement objective as a RL reward. Tradeoffs between the placement objective and alignment of macros can be controlled by a tunable alignment parameter.