公开(公告)号：US20250053444A1

公开(公告)日：2025-02-13

申请号：US18814371

申请日：2024-08-23

Applicant: Google LLC

Inventor： Jeffrey Adgate Dean ,  Sudip Roy ,  Michael Acheson Isard ,  Aakanksha Chowdhery ,  Brennan Saeta ,  Chandramohan Amyangot Thekkath ,  Daniel William Hurt ,  Hyeontaek Lim ,  Laurent El Shafey ,  Parker Edward Schuh ,  Paul Ronald Barham ,  Ruoming Pang ,  Ryan Sepassi ,  Sanjay Ghemawat ,  Yonghui Wu

IPC: G06F9/48 ,  G06N3/063 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing machine learning workloads, e.g., computations for training a neural network or computing an inference using a neural network, across multiple hardware accelerators. One of the systems comprises a plurality of accelerator islands, each hardware accelerator island comprising a respective plurality of hardware devices that include a plurality of hardware accelerators and a corresponding host for each of the plurality of hardware accelerators; and a respective scheduler for each of the accelerator islands that is configured to schedule workloads across the plurality of accelerators and corresponding hosts in the accelerator island, wherein the system is configured to: receive data representing a machine learning workload; and assign a respective portion of the machine learning workload to each of the plurality of accelerator islands for scheduling by the respective scheduler for the accelerator island.

公开(公告)号：US12086516B2

公开(公告)日：2024-09-10

申请号：US18310427

申请日：2023-05-01

Applicant: Google LLC

Inventor： Chian-min Richard Ho ,  William Hang ,  Mustafa Nazim Yazgan ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Azalia Mirhoseini ,  Emre Tuncer ,  Ya Wang ,  Anand Babu

IPC: G06F30/27 ,  G06F30/392

CPC classification number: G06F30/27 ,  G06F30/392

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US12056613B2

公开(公告)日：2024-08-06

申请号：US18199901

申请日：2023-05-19

Applicant: Google LLC

Inventor： Andrea Gesmundo ,  Jeffrey Adgate Dean

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network system to perform multiple machine learning tasks.

公开(公告)号：US12050991B1

公开(公告)日：2024-07-30

申请号：US16418074

申请日：2019-05-21

Applicant: Google LLC

Inventor： Viren Jain ,  Jeffrey Adgate Dean

IPC: G06N3/08 ,  G06N3/045 ,  G06N20/00

CPC classification number: G06N3/08 ,  G06N3/045 ,  G06N20/00

Abstract: The present disclosure provides systems and methods that generate new architectures for artificial neural networks based on connectomics data that describes connections between biological neurons of a biological organism. In particular, in some implementations, a computing system can identify one or more new artificial neural network architectures by performing a neural architecture search over a search space that is constrained based at least in part on the connectomics data.

公开(公告)号：US20240112027A1

公开(公告)日：2024-04-04

申请号：US18477546

申请日：2023-09-28

Applicant: Google LLC

Inventor： Yanqi Zhou ,  Yanping Huang ,  Yifeng Lu ,  Andrew M. Dai ,  Siamak Shakeri ,  Zhifeng Chen ,  James Laudon ,  Quoc V. Le ,  Da Huang ,  Nan Du ,  David Richard So ,  Daiyi Peng ,  Yingwei Cui ,  Jeffrey Adgate Dean ,  Chang Lan

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing neural architecture search for machine learning models. In one aspect, a method comprises receiving training data for a machine learning, generating a plurality of candidate neural networks for performing the machine learning task, wherein each candidate neural network comprises a plurality of instances of a layer block composed of a plurality of layers, for each candidate neural network, selecting a respective type for each of the plurality of layers from a set of layer types that comprises, training the candidate neural network and evaluating performance scores for the trained candidate neural networks as applied to the machine learning task, and determining a final neural network for performing the machine learning task based at least on the performance scores for the candidate neural networks.

公开(公告)号：US20240062062A1

公开(公告)日：2024-02-22

申请号：US18376362

申请日：2023-10-03

Applicant: Google LLC

Inventor： Samuel Bengio ,  Mohammad Norouzi ,  Benoit Steiner ,  Jeffrey Adgate Dean ,  Hieu Hy Pham ,  Azalia Mirhoseini ,  Quoc V. Le ,  Naveen Kumar ,  Yuefeng Zhou ,  Rasmus Munk Larsen

IPC: G06N3/08 ,  G06N5/04 ,  G06N3/10 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N5/04 ,  G06N3/105 ,  G06N3/044 ,  G06N3/045

Abstract: A method for determining a placement for machine learning model operations across multiple hardware devices is described. The method includes receiving data specifying a machine learning model to be placed for distributed processing on multiple hardware devices; generating, from the data, a sequence of operation embeddings, each operation embedding in the sequence characterizing respective operations necessary to perform the processing of the machine learning model; processing the sequence of operation embeddings using a placement recurrent neural network in accordance with first values of a plurality network parameters of the placement recurrent neural network to generate a network output that defines a placement of the operations characterized by the operation embeddings in the sequence across the plurality of devices; and scheduling the machine learning model for processing by the multiple hardware devices by placing the operations on the multiple devices according to the placement defined by the network output.

公开(公告)号：US20210334445A1

公开(公告)日：2021-10-28

申请号：US17238128

申请日：2021-04-22

Applicant: Google LLC

Inventor： Anna Darling Goldie ,  Azalia Mirhoseini ,  Ebrahim Songhori ,  Wenjie Jiang ,  Shen Wang ,  Roger David Carpenter ,  Young-Joon Lee ,  Mustafa Nazim Yazgan ,  Chian-min Richard Ho ,  Quoc V. Le ,  James Laudon ,  Jeffrey Adgate Dean ,  Kavya Srinivasa Setty ,  Omkar Pathak

IPC: G06F30/392 ,  G06F30/398 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip placement. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip placement, comprising placing a respective macro node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the macro node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US20210311909A1

公开(公告)日：2021-10-07

申请号：US17350804

申请日：2021-06-17

Applicant: Google LLC

Inventor： Yasushi Saito ,  Sanjay Ghemawat ,  Jeffrey Adgate Dean

IPC: G06F16/16 ,  G06F16/11 ,  G06F16/182 ,  G06F16/215 ,  G06F16/174

Abstract: A method for deleting obsolete files from a file system is provided. The method includes receiving a request to delete a reference to a first target file of a plurality of target files stored in a file system, the first target file having a first target file name. A first reference file whose file name includes the first target file name is identified. The first reference file is deleted from the file system. The method further includes determining whether the file system includes at least one reference file, distinct from the first reference file, whose file name includes the first target file name. In accordance with a determination that the file system does not include the at least one reference file, the first target file is deleted from the file system.

公开(公告)号：US10949881B2

公开(公告)日：2021-03-16

申请号：US15478767

申请日：2017-04-04

Applicant: Google LLC

Inventor： Jeffrey Adgate Dean ,  Georges Harik ,  Paul Buchheit

IPC: G06F16/00 ,  G06Q30/02 ,  G06F16/2455

Abstract: Targeting information (also referred to as ad “serving constraints”) or candidate targeting information for an advertisement is identified. Targeting information may be identified by extracting topics or concepts from, and/or generating topics or concepts based on, ad information, such as information from a Web page to which an ad is linked (or some other Web page of interest to the ad or advertiser). The topics or concepts may be relevant queries associated with the Web page of interest, clusters, etc.

公开(公告)号：US20210019604A1

公开(公告)日：2021-01-21

申请号：US16940131

申请日：2020-07-27

Applicant: Google LLC

Inventor： Gregory Sean Corrado ,  Ilya Sutskever ,  Jeffrey Adgate Dean

IPC: G06N3/04 ,  G16H50/20 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.