公开(公告)号：US12205038B2

公开(公告)日：2025-01-21

申请号：US18321691

申请日：2023-05-22

Applicant: Google LLC

Inventor： Yanqi Zhou ,  Sudip Roy ,  Amirali Abdolrashidi ,  Daniel Lin-Kit Wong ,  Chao Ma ,  Qiumin Xu ,  Azalia Mirhoseini

IPC: G06N3/04 ,  G06F18/20 ,  G06F18/21 ,  G06F18/2115 ,  G06N3/045 ,  G06N3/049 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing the execution of the operations of a neural network. One of the methods includes obtaining data representing a graph characterizing a plurality of operations of a neural network, wherein each node of the graph characterizes an operation of the neural network and each edge of the graph characterizes data dependency between the operations; processing the data representing the graph using a graph embedding neural network to generate an embedding of the graph; and processing the embedding of the graph using a policy neural network to generate a task output, wherein the task output comprises, for each of the plurality of operations of the neural network, a respective decision for a particular optimization task.

公开(公告)号：US12067476B2

公开(公告)日：2024-08-20

申请号：US18244171

申请日：2023-09-08

Applicant: Google LLC

Inventor： Noam M. Shazeer ,  Azalia Mirhoseini ,  Krzysztof Stanislaw Maziarz

IPC: G06N3/045 ,  G06N3/08

CPC classification number: G06N3/045 ,  G06N3/08

Abstract: A system includes a neural network that includes a Mixture of Experts (MoE) subnetwork between a first neural network layer and a second neural network layer. The MoE subnetwork includes multiple expert neural networks. Each expert neural network is configured to process a first layer output generated by the first neural network layer to generate a respective expert output. The MoE subnetwork further includes a gating subsystem that selects, based on the first layer output, one or more of the expert neural networks and determine a respective weight for each selected expert neural network, provides the first layer output as input to each of the selected expert neural networks, combines the expert outputs generated by the selected expert neural networks in accordance with the weights for the selected expert neural networks to generate an MoE output, and provides the MoE output as input to the second neural network layer.

公开(公告)号：US11853677B2

公开(公告)日：2023-12-26

申请号：US18082392

申请日：2022-12-15

Applicant: Google LLC

Inventor： Anna Darling Goldie ,  Azalia Mirhoseini ,  Ebrahim Songhori ,  Wenjie Jiang ,  Shen Wang ,  Roger David Carpenter ,  Young-Joon Lee ,  Mustafa Nazim Yazgan ,  Chian-min Richard Ho ,  Quoc V. Le ,  James Laudon ,  Jeffrey Adgate Dean ,  Kavya Srinivasa Setty ,  Omkar Pathak

IPC: G06F30/392 ,  G06F30/398 ,  G06N3/08

CPC classification number: G06F30/392 ,  G06F30/398 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip placement. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip placement, comprising placing a respective macro node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the macro node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US20230394203A1

公开(公告)日：2023-12-07

申请号：US18310427

申请日：2023-05-01

Applicant: Google LLC

Inventor： Chian-min Richard Ho ,  William Hang ,  Mustafa Nazim Yazgan ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Azalia Mirhoseini ,  Emre Tuncer ,  Ya Wang ,  Anand Babu

IPC: G06F30/27 ,  G06F30/392

CPC classification number: G06F30/27 ,  G06F30/392

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US11100266B2

公开(公告)日：2021-08-24

申请号：US16889130

申请日：2020-06-01

Applicant: Google LLC

Inventor： Chian-min Richard Ho ,  William Hang ,  Mustafa Nazim Yazgan ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Azalia Mirhoseini ,  Emre Tuncer ,  Ya Wang ,  Anand Babu

IPC: G06F30/00 ,  G06F30/30 ,  G06F30/27 ,  G06F30/392

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US20240370693A1

公开(公告)日：2024-11-07

申请号：US18285578

申请日：2022-04-06

Applicant: Google LLC

Inventor： Dan Zhang ,  Safeen Huda ,  Azalia Mirhoseini ,  Anna Darling Goldie ,  Ebrahim Songhori

IPC: G06N3/04 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining a hardware datapath for a hardware accelerator computer chip.

公开(公告)号：US11803747B2

公开(公告)日：2023-10-31

申请号：US16878720

申请日：2020-05-20

Applicant: Google LLC

Inventor： Samuel Bengio ,  Mohammad Norouzi ,  Benoit Steiner ,  Jeffrey Adgate Dean ,  Hieu Hy Pham ,  Azalia Mirhoseini ,  Quoc V. Le ,  Naveen Kumar ,  Yuefeng Zhou ,  Rasmus Munk Larsen

IPC: G06N3/08 ,  G06N5/04 ,  G06N3/10 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N3/044 ,  G06N3/045 ,  G06N3/105 ,  G06N5/04

Abstract: A method for determining a placement for machine learning model operations across multiple hardware devices is described. The method includes receiving data specifying a machine learning model to be placed for distributed processing on multiple hardware devices; generating, from the data, a sequence of operation embeddings, each operation embedding in the sequence characterizing respective operations necessary to perform the processing of the machine learning model; processing the sequence of operation embeddings using a placement recurrent neural network in accordance with first values of a plurality network parameters of the placement recurrent neural network to generate a network output that defines a placement of the operations characterized by the operation embeddings in the sequence across the plurality of devices; and scheduling the machine learning model for processing by the multiple hardware devices by placing the operations on the multiple devices according to the placement defined by the network output.

公开(公告)号：US11790214B2

公开(公告)日：2023-10-17

申请号：US16879187

申请日：2020-05-20

Applicant: Google LLC

Inventor： Noam M. Shazeer ,  Azalia Mirhoseini ,  Krzysztof Stanislaw Maziarz

IPC: G06N3/045 ,  G06N3/08

CPC classification number: G06N3/045 ,  G06N3/08

Abstract: A system includes a neural network that includes a Mixture of Experts (MoE) subnetwork between a first neural network layer and a second neural network layer. The MoE subnetwork includes multiple expert neural networks. Each expert neural network is configured to process a first layer output generated by the first neural network layer to generate a respective expert output. The MoE subnetwork further includes a gating subsystem that selects, based on the first layer output, one or more of the expert neural networks and determine a respective weight for each selected expert neural network, provides the first layer output as input to each of the selected expert neural networks, combines the expert outputs generated by the selected expert neural networks in accordance with the weights for the selected expert neural networks to generate an MoE output, and provides the MoE output as input to the second neural network layer.

公开(公告)号：US20230306266A1

公开(公告)日：2023-09-28

申请号：US18321691

申请日：2023-05-22

Applicant: Google LLC

Inventor： Yanqi Zhou ,  Sudip Roy ,  Amirali Abdolrashidi ,  Daniel Lin-Kit Wong ,  Chao Ma ,  Qiumin Xu ,  Azalia Mirhoseini

IPC: G06N3/084 ,  G06N3/049 ,  G06F18/20 ,  G06F18/21 ,  G06F18/2115 ,  G06N3/045

CPC classification number: G06N3/084 ,  G06N3/049 ,  G06F18/29 ,  G06F18/217 ,  G06F18/2115 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing the execution of the operations of a neural network. One of the methods includes obtaining data representing a graph characterizing a plurality of operations of a neural network, wherein each node of the graph characterizes an operation of the neural network and each edge of the graph characterizes data dependency between the operations; processing the data representing the graph using a graph embedding neural network to generate an embedding of the graph; and processing the embedding of the graph using a policy neural network to generate a task output, wherein the task output comprises, for each of the plurality of operations of the neural network, a respective decision for a particular optimization task.

公开(公告)号：US11675940B2

公开(公告)日：2023-06-13

申请号：US17409566

申请日：2021-08-23

Applicant: Google LLC

Inventor： Chian-Min Richard Ho ,  William Hang ,  Mustafa Nazim Yazgan ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Azalia Mirhoseini ,  Emre Tuncer ,  Ya Wang ,  Anand Babu

IPC: G06F30/27 ,  G06F30/392

CPC classification number: G06F30/27 ,  G06F30/392

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.