公开(公告)号：US20230376755A1

公开(公告)日：2023-11-23

申请号：US18199901

申请日：2023-05-19

Applicant: Google LLC

Inventor： Andrea Gesmundo ,  Jeffrey Adgate Dean

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network system to perform multiple machine learning tasks.

公开(公告)号：US11790216B2

公开(公告)日：2023-10-17

申请号：US16940131

申请日：2020-07-27

Applicant: Google LLC

Inventor： Gregory Sean Corrado ,  Ilya Sutskever ,  Jeffrey Adgate Dean

IPC: G06N3/08 ,  G06N3/047 ,  G16H50/20 ,  G06N3/042 ,  G06N3/044 ,  G06N3/063 ,  G06N3/02

CPC classification number: G06N3/047 ,  G06N3/042 ,  G06N3/044 ,  G06N3/063 ,  G16H50/20 ,  G06N3/02 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.

公开(公告)号：US11775480B2

公开(公告)日：2023-10-03

申请号：US17350804

申请日：2021-06-17

Applicant: Google LLC

Inventor： Yasushi Saito ,  Sanjay Ghemawat ,  Jeffrey Adgate Dean

IPC: G06F16/16 ,  G06F16/11 ,  G06F16/182 ,  G06F16/215 ,  G06F16/174

CPC classification number: G06F16/162 ,  G06F16/11 ,  G06F16/1748 ,  G06F16/182 ,  G06F16/215

Abstract: A method for deleting obsolete files from a file system is provided. The method includes receiving a request to delete a reference to a first target file of a plurality of target files stored in a file system, the first target file having a first target file name. A first reference file whose file name includes the first target file name is identified. The first reference file is deleted from the file system. The method further includes determining whether the file system includes at least one reference file, distinct from the first reference file, whose file name includes the first target file name. In accordance with a determination that the file system does not include the at least one reference file, the first target file is deleted from the file system.

公开(公告)号：US11763146B1

公开(公告)日：2023-09-19

申请号：US16986974

申请日：2020-08-06

Applicant: Google LLC

Inventor： Yuan Yu ,  Jeffrey Adgate Dean

IPC: G06N3/08 ,  G06N3/044

CPC classification number: G06N3/08 ,  G06N3/044

Abstract: Systems and methods for processing loops in computational graphs representing machine learning models are disclosed. An example method begins with obtaining data representing a computational graph. Data identifying an allocation of the computational graph across devices is obtained. Additionally, one or more nodes in the computational graph that represent a respective control flow statement are identified. For each identified node, a structure of nodes and edges that represents an operation that provides a current state of recursion or iteration in the respective control flow statement is generated. This structure is inserted into the computational graph and the allocation of nodes to devices is modified to assign the structure to a device.

公开(公告)号：US20230162010A1

公开(公告)日：2023-05-25

申请号：US17532572

申请日：2021-11-22

Applicant: Google LLC

Inventor： Azalia Mirhoseini ,  Safeen Huda ,  Martin Christoph Maas ,  Paras Jagdish Jain ,  Jeffrey Adgate Dean

IPC: G06N3/063 ,  G06F15/80 ,  G06F11/34 ,  G06F11/30

CPC classification number: G06N3/063 ,  G06F15/8046 ,  G06F11/3409 ,  G06F11/3062 ,  G06F11/3024

Abstract: Systems and methods are provided for designing approximate, low-power deep learning accelerator chips that have little to no accuracy loss when executing a deep learning model. A set of approximate systolic arrays may be generated. The performance of each approximate systolic array in the set of approximate systolic arrays processing a deep neural network (DNN) may be determined. Each layer in the DNN may be mapped to an approximate systolic array in the set of approximate systolic arrays. A subset of the set of approximate systolic arrays may be selected for inclusion in the inference chip design based on the mapping and the performance of each approximate systolic array in the set of approximate systolic arrays.

公开(公告)号：US20230118303A1

公开(公告)日：2023-04-20

申请号：US18082415

申请日：2022-12-15

Applicant: Google LLC

Inventor： Jeffrey Adgate Dean ,  Sudip Roy ,  Michael Acheson Isard ,  Aakanksha Chowdhery ,  Brennan Saeta ,  Chandramohan Amyangot Thekkath ,  Daniel William Hurt ,  Hyeontaek Lim ,  Laurent El Shafey ,  Parker Edward Schuh ,  Paul Ronald Barham ,  Ruoming Pang ,  Ryan Sepassi ,  Sanjay Ghemawat ,  Yonghui Wu

IPC: G06F9/48 ,  G06N3/063 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing machine learning workloads, e.g., computations for training a neural network or computing an inference using a neural network, across multiple hardware accelerators. One of the systems comprises a plurality of accelerator islands, each hardware accelerator island comprising a respective plurality of hardware devices that include a plurality of hardware accelerators and a corresponding host for each of the plurality of hardware accelerators; and a respective scheduler for each of the accelerator islands that is configured to schedule workloads across the plurality of accelerators and corresponding hosts in the accelerator island, wherein the system is configured to: receive data representing a machine learning workload; and assign a respective portion of the machine learning workload to each of the plurality of accelerator islands for scheduling by the respective scheduler for the accelerator island.

公开(公告)号：US20230117786A1

公开(公告)日：2023-04-20

申请号：US18082392

申请日：2022-12-15

Applicant: Google LLC

Inventor： Anna Darling Goldie ,  Azalia Mirhoseini ,  Ebrahim Songhori ,  Wenjie Jiang ,  Shen Wang ,  Roger David Carpenter ,  Young-Joon Lee ,  Mustafa Nazim Yazgan ,  Chian-min Richard Ho ,  Quoc V. Le ,  James Laudon ,  Jeffrey Adgate Dean ,  Kavya Srinivasa Setty ,  Omkar Pathak

IPC: G06F30/392 ,  G06F30/398 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip placement. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip placement, comprising placing a respective macro node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the macro node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US11455514B2

公开(公告)日：2022-09-27

申请号：US16554217

申请日：2019-08-28

Applicant: Google LLC

Inventor： Benoit Steiner ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Hieu Hy Pham ,  Azalia Mirhoseini ,  Quoc V. Le

IPC: G06N3/04 ,  G06F9/50 ,  G06N20/00 ,  G06F16/901 ,  G06N3/063 ,  G06N3/08 ,  G06N5/04

Abstract: A method for determining a placement for machine learning model operations across multiple hardware devices includes receiving data specifying machine learning operations, and determining a placement that assigns each of the operations specified by the data to a respective device from the multiple hardware devices. Determining the placement includes: generating, from the data, a respective operation embedding for each of the operations; grouping the operations into multiple operation groups, comprising processing each of the respective operation embeddings using a grouper neural network having multiple grouper parameters, in which the grouper neural network is configured to, for each of the operations, process the operation embedding for the operation in accordance with first values of the grouper parameters to generate a grouper output that assigns the operation to an operation group from the multiple operation groups; and assigning each of the operation groups to a respective device from the multiple hardware devices.

公开(公告)号：US11423337B2

公开(公告)日：2022-08-23

申请号：US16841859

申请日：2020-04-07

Applicant: Google LLC

Inventor： Oriol Vinyals ,  Jeffrey Adgate Dean ,  Geoffrey E. Hinton

IPC: G06N20/20 ,  G06N7/00 ,  G06N20/00 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a distilled machine learning model. One of the methods includes training a cumbersome machine learning model, wherein the cumbersome machine learning model is configured to receive an input and generate a respective score for each of a plurality of classes; and training a distilled machine learning model on a plurality of training inputs, wherein the distilled machine learning model is also configured to receive inputs and generate scores for the plurality of classes, comprising: processing each training input using the cumbersome machine learning model to generate a cumbersome target soft output for the training input; and training the distilled machine learning model to, for each of the training inputs, generate a soft output that matches the cumbersome target soft output for the training input.

公开(公告)号：US20220222219A1

公开(公告)日：2022-07-14

申请号：US17671068

申请日：2022-02-14

Applicant: Google LLC

Inventor： Jeffrey Adgate Dean ,  Sanjay Ghemawat ,  Andrew Fikes ,  Yasushi Saito

IPC: G06F16/182 ,  G06F16/22 ,  G06F9/50 ,  G06F16/13 ,  H04L67/1001

Abstract: A method of accessing data includes storing a table that includes a plurality of tablets corresponding to distinct non-overlapping table portions. Respective pluralities of tablet access objects and application objects are stored in a plurality of servers. A distinct application object and distinct tablet are associated with each tablet access object. Each application object corresponds to a distinct instantiation of an application associated with the table. The tablet access objects and associated application objects are redistributed among the servers in accordance with a first load-balancing criterion. A first request directed to a respective tablet is received from a client. In response, the tablet access object associated with the respective tablet is used to perform a data access operation on the respective tablet, and the application object associated with the respective tablet is used to perform an additional computational operation to produce a result to be returned to the client.