公开(公告)号：US11216609B2

公开(公告)日：2022-01-04

申请号：US17238128

申请日：2021-04-22

Applicant: Google LLC

Inventor： Anna Darling Goldie ,  Azalia Mirhoseini ,  Ebrahim Songhori ,  Wenjie Jiang ,  Shen Wang ,  Roger David Carpenter ,  Young-Joon Lee ,  Mustafa Nazim Yazgan ,  Chian-Min Richard Ho ,  Quoc V. Le ,  James Laudon ,  Jeffrey Adgate Dean ,  Kavya Srinivasa Setty ,  Omkar Pathak

IPC: G06F30/392 ,  G06N3/08 ,  G06F30/398

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip placement. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip placement, comprising placing a respective macro node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the macro node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US10860925B2

公开(公告)日：2020-12-08

申请号：US15337744

申请日：2016-10-28

Applicant: Google LLC

Inventor： Paul A. Tucker ,  Jeffrey Adgate Dean ,  Sanjay Ghemawat ,  Yuan Yu

IPC: G06N3/08 ,  G06F9/50 ,  G06N3/063 ,  G06N3/04 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for receiving a request from a client to process a computational graph; obtaining data representing the computational graph, the computational graph comprising a plurality of nodes and directed edges, wherein each node represents a respective operation, wherein each directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node; identifying a plurality of available devices for performing the requested operation; partitioning the computational graph into a plurality of subgraphs, each subgraph comprising one or more nodes in the computational graph; and assigning, for each subgraph, the operations represented by the one or more nodes in the subgraph to a respective available device in the plurality of available devices for operation.

公开(公告)号：US20200302302A1

公开(公告)日：2020-09-24

申请号：US16898971

申请日：2020-06-11

Applicant: Google LLC

Inventor： Paul A. Tucker ,  Jeffrey Adgate Dean ,  Sanjay Ghemawat ,  Yuan Yu

IPC: G06N3/08 ,  G06F9/50 ,  G06N3/063 ,  G06N3/04 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for receiving a request from a client to process a computational graph; obtaining data representing the computational graph, the computational graph comprising a plurality of nodes and directed edges, wherein each node represents a respective operation, wherein each directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node; identifying a plurality of available devices for performing the requested operation; partitioning the computational graph into a plurality of subgraphs, each subgraph comprising one or more nodes in the computational graph; and assigning, for each subgraph, the operations represented by the one or more nodes in the subgraph to a respective available device in the plurality of available devices for operation.

公开(公告)号：US10726327B2

公开(公告)日：2020-07-28

申请号：US15588535

申请日：2017-05-05

Applicant: Google LLC

Inventor： Gregory Sean Corrado ,  Ilya Sutskever ,  Jeffrey Adgate Dean

IPC: G06N3/08 ,  G06N3/04 ,  G16H50/20 ,  G06N3/063 ,  G06N3/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.

公开(公告)号：US20200234192A1

公开(公告)日：2020-07-23

申请号：US16841859

申请日：2020-04-07

Applicant: Google LLC

Inventor： Oriol Vinyals ,  Jeffrey Adgate Dean ,  Geoffrey E. Hinton

IPC: G06N20/00 ,  G06N7/00 ,  G06N3/04 ,  G06N20/20 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a distilled machine learning model. One of the methods includes training a cumbersome machine learning model, wherein the cumbersome machine learning model is configured to receive an input and generate a respective score for each of a plurality of classes; and training a distilled machine learning model on a plurality of training inputs, wherein the distilled machine learning model is also configured to receive inputs and generate scores for the plurality of classes, comprising: processing each training input using the cumbersome machine learning model to generate a cumbersome target soft output for the training input; and training the distilled machine learning model to, for each of the training inputs, generate a soft output that matches the cumbersome target soft output for the training input.

公开(公告)号：US10699043B2

公开(公告)日：2020-06-30

申请号：US16703837

申请日：2019-12-04

Applicant: Google LLC

Inventor： Chian-min Richard Ho ,  William Hang ,  Mustafa Nazim Yazgan ,  Anna Darling Goldie ,  Jeffrey Adgate Dean ,  Azalia Mirhoseini ,  Emre Tuncer ,  Ya Wang ,  Anand Babu

IPC: G06F17/50 ,  G06F30/27 ,  G06F30/392

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.

公开(公告)号：US10692003B2

公开(公告)日：2020-06-23

申请号：US16445330

申请日：2019-06-19

Applicant: Google LLC

Inventor： Samuel Bengio ,  Mohammad Norouzi ,  Benoit Steiner ,  Jeffrey Adgate Dean ,  Hieu Hy Pham ,  Azalia Mirhoseini ,  Quoc V. Le ,  Naveen Kumar ,  Yuefeng Zhou ,  Rasmus Munk Larsen

IPC: G06N3/08 ,  G06N5/04 ,  G06N3/04 ,  G06N3/10

Abstract: A method for determining a placement for machine learning model operations across multiple hardware devices is described. The method includes receiving data specifying a machine learning model to be placed for distributed processing on multiple hardware devices; generating, from the data, a sequence of operation embeddings, each operation embedding in the sequence characterizing respective operations necessary to perform the processing of the machine learning model; processing the sequence of operation embeddings using a placement recurrent neural network in accordance with first values of a plurality network parameters of the placement recurrent neural network to generate a network output that defines a placement of the operations characterized by the operation embeddings in the sequence across the plurality of devices; and scheduling the machine learning model for processing by the multiple hardware devices by placing the operations on the multiple devices according to the placement defined by the network output.

公开(公告)号：US20190220781A1

公开(公告)日：2019-07-18

申请号：US16368526

申请日：2019-03-28

Applicant: Google LLC

Inventor： Oriol Vinyals ,  Jeffrey Adgate Dean ,  Geoffrey E. Hinton

IPC: G06N20/00 ,  G06N3/04 ,  G06N7/00 ,  G06N20/20

CPC classification number: G06N20/00 ,  G06N3/0454 ,  G06N3/084 ,  G06N7/00 ,  G06N7/005 ,  G06N20/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a distilled machine learning model. One of the methods includes training a cumbersome machine learning model, wherein the cumbersome machine learning model is configured to receive an input and generate a respective score for each of a plurality of classes; and training a distilled machine learning model on a plurality of training inputs, wherein the distilled machine learning model is also configured to receive inputs and generate scores for the plurality of classes, comprising: processing each training input using the cumbersome machine learning model to generate a cumbersome target soft output for the training input; and training the distilled machine learning model to, for each of the training inputs, generate a soft output that matches the cumbersome target soft output for the training input.

公开(公告)号：US20190156367A1

公开(公告)日：2019-05-23

申请号：US16258939

申请日：2019-01-28

Applicant: Google LLC

Inventor： Jeffrey Adgate Dean ,  Georges Harik ,  Paul Buchheit

IPC: G06Q30/02 ,  G06F16/951 ,  G06F16/35

Abstract: The relevance of advertisements to a user's interests is improved. In one implementation, the content of a web page is analyzed to determine a list of one or more topics associated with that web page. An advertisement is considered to be relevant to that web page if it is associated with keywords belonging to the list of one or more topics. One or more of these relevant advertisements may be provided for rendering in conjunction with the web page or related web pages.

公开(公告)号：US20180341991A1

公开(公告)日：2018-11-29

申请号：US16055007

申请日：2018-08-03

Applicant: GOOGLE LLC

Inventor： Jeffrey Adgate Dean ,  Krishna Bharat ,  Paul Buchheit

IPC: G06Q30/02 ,  G06F17/30

CPC classification number: G06Q30/0277 ,  G06F16/951 ,  G06F16/9535 ,  G06Q30/02

Abstract: The usefulness of content (target content), such as advertisements, may be increased by determining additional content and providing such additional content in association with the content. The target content may be text, a Web page, a URL, a search query, etc. The additional content might be related suggested queries (e.g. “Try a search for ______”), news articles (or excerpts or summaries thereof), reviews (or excerpts or summaries thereof), advertisements, user group messages, etc.