公开(公告)号：US20200341950A1

公开(公告)日：2020-10-29

申请号：US16927264

申请日：2020-07-13

Applicant: Google LLC

Inventor： Jeffrey Dean ,  Sanjay Ghemawat ,  Andrew Fikes ,  Yasushi Saito

IPC: G06F16/182 ,  G06F16/22 ,  G06F9/50 ,  G06F16/13 ,  H04L29/08

Abstract: A method of accessing data includes storing a table that includes a plurality of tablets corresponding to distinct non-overlapping table portions. Respective pluralities of tablet access objects and application objects are stored in a plurality of servers. A distinct application object and distinct tablet are associated with each tablet access object. Each application object corresponds to a distinct instantiation of an application associated with the table. The tablet access objects and associated application objects are redistributed among the servers in accordance with a first load-balancing criterion. A first request directed to a respective tablet is received from a client. In response, the tablet access object associated with the respective tablet is used to perform a data access operation on the respective tablet, and the application object associated with the respective tablet is used to perform an additional computational operation to produce a result to be returned to the client.

公开(公告)号：US20200302302A1

公开(公告)日：2020-09-24

申请号：US16898971

申请日：2020-06-11

Applicant: Google LLC

Inventor： Paul A. Tucker ,  Jeffrey Adgate Dean ,  Sanjay Ghemawat ,  Yuan Yu

IPC: G06N3/08 ,  G06F9/50 ,  G06N3/063 ,  G06N3/04 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for receiving a request from a client to process a computational graph; obtaining data representing the computational graph, the computational graph comprising a plurality of nodes and directed edges, wherein each node represents a respective operation, wherein each directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node; identifying a plurality of available devices for performing the requested operation; partitioning the computational graph into a plurality of subgraphs, each subgraph comprising one or more nodes in the computational graph; and assigning, for each subgraph, the operations represented by the one or more nodes in the subgraph to a respective available device in the plurality of available devices for operation.

公开(公告)号：US20200293292A1

公开(公告)日：2020-09-17

申请号：US16834191

申请日：2020-03-30

Applicant: Google LLC

Inventor： Gautham Thambidorai ,  Matthew Rosencrantz ,  Sanjay Ghemawat ,  Srdjan Petrovic ,  Ivan Posva

IPC: G06F8/35 ,  G06F8/41 ,  G06F16/901 ,  G06F8/30

Abstract: A programming model provides a method for type inference in programming operations. Information defining one or more attributes of an operation is received, the information specifying a field including a field name and a field type identifier for each of the attributes. Constraints for the operation are determined at least based on the attributes, wherein the constraints restrict at least one of a type of input for the operation or a type of output for the operation. Information defining an input for the operation is received, and it is determined, based on the constraints and the received information defining the input, the type of output for the operation. The type of output is associated with an output for the operation.

公开(公告)号：US10740301B2

公开(公告)日：2020-08-11

申请号：US15868928

申请日：2018-01-11

Applicant: Google LLC

Inventor： Jeffrey A. Dean ,  Sanjay Ghemawat ,  Andrew B. Fikes ,  Yasushi Saito

IPC: G06F16/182 ,  G06F16/22 ,  G06F9/50 ,  G06F16/13 ,  H04L29/08

Abstract: A method of accessing data includes storing a table that includes a plurality of tablets corresponding to distinct non-overlapping table portions. Respective pluralities of tablet access objects and application objects are stored in a plurality of servers. A distinct application object and distinct tablet are associated with each tablet access object. Each application object corresponds to a distinct instantiation of an application associated with the table. The tablet access objects and associated application objects are redistributed among the servers in accordance with a first load-balancing criterion. A first request directed to a respective tablet is received from a client. In response, the tablet access object associated with the respective tablet is used to perform a data access operation on the respective tablet, and the application object associated with the respective tablet is used to perform an additional computational operation to produce a result to be returned to the client.

公开(公告)号：US20190272264A1

公开(公告)日：2019-09-05

申请号：US16417126

申请日：2019-05-20

Applicant: Google LLC

Inventor： Jeffrey Dean ,  Sanjay Ghemawat

IPC: G06F16/22 ,  G06F16/2453 ,  G06F9/54 ,  G06F9/48 ,  G06F16/23

Abstract: A method performs large-scale data processing in a distributed and parallel processing environment. The method defines application-independent map and reduce operations, each invoking one or more library functions that automatically handle data partitioning, parallelization of computations, and fault tolerance. A user specifies a map operation, which calls one or more of the application-independent map operators to perform data read and write operations. A user also specifies a reduce operation, which calls one or more of the application-independent reduce operators to perform data read and write operations. The method executes application-independent map worker processes. Each map worker process executes the user-specified map operation to read designated portions of input files and store intermediate data values in intermediate data structures. The method also executes application-independent reduce worker processes. Each reduce worker process executes the user-specified reduce operation to read intermediate data values from the intermediate data structures and produce final output data.