公开(公告)号：US20080098375A1

公开(公告)日：2008-04-24

申请号：US11537514

申请日：2006-09-29

Applicant: Michael A. Isard

Inventor： Michael A. Isard

IPC: G06F9/45

CPC classification number: G06F9/5066 ,  G06F11/1438

Abstract: A general purpose high-performance distributed execution engine for coarse-grained data-parallel applications is proposed that allows developers to easily create large-scale distributed applications without requiring them to master concurrency techniques beyond being able to draw a graph of the data-dependencies of their algorithms. Based on the graph, a job manager intelligently distributes the work load so that the resources of the execution engine are used efficiently. During runtime, the job manager (or other entity) can automatically modify the graph to improve efficiency. The modifications are based on runtime information, topology of the distributed execution engine, and/or the distributed application represented by the graph.

Abstract translation: 提出了一种用于粗粒度数据并行应用程序的通用高性能分布式执行引擎，允许开发人员轻松创建大规模分布式应用程序，而不需要它们掌握并发技术，除了能够绘制数据依赖关系的图形 他们的算法。 基于该图，作业管理器智能地分配工作负载，以便有效地使用执行引擎的资源。 在运行时，作业管理器（或其他实体）可以自动修改图形以提高效率。 这些修改基于运行时信息，分布式执行引擎的拓扑和/或由图表表示的分布式应用程序。

公开(公告)号：US20130304744A1

公开(公告)日：2013-11-14

申请号：US13468726

申请日：2012-05-10

Applicant: Frank D. McSherry ,  Rebecca Isaacs ,  Michael A. Isard ,  Derek G. Murray

Inventor： Frank D. McSherry ,  Rebecca Isaacs ,  Michael A. Isard ,  Derek G. Murray

IPC: G06F17/30

CPC classification number: G06F17/30516 ,  G06F17/30554 ,  G06F17/30958

Abstract: The techniques discussed herein efficiently perform data-parallel computations on collections of data by implementing a differential dataflow model that performs computations on differences in the collections of data. The techniques discussed herein describe defined operators for use in a data-parallel program that performs the computations on the determined differences between the collections of data by creating a lattice and indexing the differences in the collection of data according to the lattice.

Abstract translation: 本文讨论的技术通过实现对数据集合中的差异进行计算的差分数据流模型，有效地对数据集合执行数据并行计算。 本文讨论的技术描述了用于数据并行程序中的定义的操作符，其通过创建格子并根据格子索引数据收集的差异来对数据集合之间确定的差异进行计算。

公开(公告)号：US08239847B2

公开(公告)日：2012-08-07

申请号：US12406842

申请日：2009-03-18

Applicant: Yuan Yu ,  Pradeep Kumar Gunda ,  Michael A Isard

Inventor： Yuan Yu ,  Pradeep Kumar Gunda ,  Michael A Isard

IPC: G06F9/45

CPC classification number: H04L12/44 ,  G06F8/456 ,  G06F9/5066

Abstract: General-purpose distributed data-parallel computing using high-level computing languages is described. Data parallel portions of a sequential program written in a high-level language are automatically translated into a distributed execution plan. Map and reduction computations are automatically added to the plan. Patterns in the sequential program can be automatically identified to trigger map and reduction processing. Direct invocation of map and reduction processing is also provided. One or more portions of the reduce computation are pushed to the map stage and dynamic aggregation is inserted when possible. The system automatically identifies opportunities for partial reductions and aggregation, but also provides a set of extensions in a high-level computing language for the generation and optimization of the distributed execution plan. The extensions include annotations to declare functions suitable for these optimizations.

Abstract translation: 描述了使用高级计算语言的通用分布式数据并行计算。 以高级语言编写的顺序程序的数据并行部分将自动转换为分布式执行计划。 地图和缩小计算将自动添加到计划中。 顺序程序中的模式可以自动识别，以触发地图和缩小处理。 还提供了直接调用地图和缩小处理。 减少计算的一个或多个部分被推送到地图阶段，并且尽可能地插入动态聚合。 系统自动识别部分缩减和聚合的机会，但也提供了一组用于生成和优化分布式执行计划的高级计算语言的扩展。 扩展包括用于声明适合这些优化的函数的注释。

公开(公告)号：US08209664B2

公开(公告)日：2012-06-26

申请号：US12406826

申请日：2009-03-18

Applicant: Yuan Yu ,  Ulfar Erlingsson ,  Michael A Isard ,  Frank McSherry

Inventor： Yuan Yu ,  Ulfar Erlingsson ,  Michael A Isard ,  Frank McSherry

IPC: G06F9/44

CPC classification number: H04L12/44 ,  G06F8/314 ,  G06F9/5066

Abstract: General-purpose distributed data-parallel computing using high-level computing languages is described. Data parallel portions of a sequential program that is written by a developer in a high-level language are automatically translated into a distributed execution plan. A set of extensions to a sequential high-level computing language are provided to support distributed parallel computations and to facilitate generation and optimization of distributed execution plans. The extensions are fully integrated with the programming language, thereby enabling developers to write sequential language programs using known constructs while providing the ability to invoke the extensions to enable better generation and optimization of the execution plan for a distributed computing environment.

Abstract translation: 描述了使用高级计算语言的通用分布式数据并行计算。 由开发者以高级语言编写的顺序程序的数据并行部分将自动转换为分布式执行计划。 提供了一组连续高级计算语言的扩展，以支持分布式并行计算，并促进分布式执行计划的生成和优化。 扩展与编程语言完全集成，从而使开发人员可以使用已知构造编写顺序语言程序，同时提供调用扩展的能力，以实现更好的生成和优化分布式计算环境的执行计划。

公开(公告)号：US08201142B2

公开(公告)日：2012-06-12

申请号：US11537529

申请日：2006-09-29

Applicant: Michael A. Isard ,  Andrew D. Birrell ,  Yuan Yu

Inventor： Michael A. Isard ,  Andrew D. Birrell ,  Yuan Yu

IPC: G06F9/44

CPC classification number: G06T11/206

Abstract: A general purpose high-performance distributed execution engine can be used by developers to deploy large-scale distributed applications. To allow developers to easily make use of the distributed execution engine, a graph building language is proposed that enables developers to efficiently create graphs (e.g., direct acyclic graphs) that describe the subprograms to be executed and the flow of data between them. A job manager (or other appropriate entity) reads the description of the graph created with the graph building language, builds the graph based on that description, and intelligently distributes the subprograms according to the graph so that system resources are used efficiently. In one embodiment, the graph building language (and, thus, the description of the graph) includes syntax for replication, pointwise connect, cross connect and merge.

Abstract translation: 开发人员可以使用通用的高性能分布式执行引擎来部署大型分布式应用程序。 为了允许开发人员轻松利用分布式执行引擎，提出了一种图形构建语言，使开发人员能够有效地创建描述要执行的子程序和它们之间的数据流的图形（例如，直接非循环图）。 作业管理器（或其他适当的实体）读取使用图形构建语言创建的图形的描述，根据该描述构建图形，并根据图形智能地分配子程序，以便有效地使用系统资源。 在一个实施例中，图形构建语言（以及因此，图形的描述）包括用于复制，点连接，交叉连接和合并的语法。

公开(公告)号：US20100241827A1

公开(公告)日：2010-09-23

申请号：US12406826

申请日：2009-03-18

Applicant: Yuan Yu ,  Ulfar Erlingsson ,  Michael A. Isard ,  Frank McSherry

Inventor： Yuan Yu ,  Ulfar Erlingsson ,  Michael A. Isard ,  Frank McSherry

IPC: G06F15/76 ,  G06F9/06

CPC classification number: H04L12/44 ,  G06F8/314 ,  G06F9/5066

Abstract: General-purpose distributed data-parallel computing using high-level computing languages is described. Data parallel portions of a sequential program that is written by a developer in a high-level language are automatically translated into a distributed execution plan. A set of extensions to a sequential high-level computing language are provided to support distributed parallel computations and to facilitate generation and optimization of distributed execution plans. The extensions are fully integrated with the programming language, thereby enabling developers to write sequential language programs using known constructs while providing the ability to invoke the extensions to enable better generation and optimization of the execution plan for a distributed computing environment.

Abstract translation: 描述了使用高级计算语言的通用分布式数据并行计算。 由开发者以高级语言编写的顺序程序的数据并行部分将自动转换为分布式执行计划。 提供了一组连续高级计算语言的扩展，以支持分布式并行计算，并促进分布式执行计划的生成和优化。 扩展与编程语言完全集成，从而使开发人员可以使用已知构造编写顺序语言程序，同时提供调用扩展的能力，以实现更好的生成和优化分布式计算环境的执行计划。

公开(公告)号：US20080082644A1

公开(公告)日：2008-04-03

申请号：US11537506

申请日：2006-09-29

Applicant: Michael A. Isard ,  Andrew D. Birrell ,  Mihai-Dan Budiu ,  Yuan Yu ,  Dennis C. Fetterly

Inventor： Michael A. Isard ,  Andrew D. Birrell ,  Mihai-Dan Budiu ,  Yuan Yu ,  Dennis C. Fetterly

IPC: G06F15/173

CPC classification number: H04L67/1097 ,  G06F9/5066

Abstract: A general purpose high-performance distributed execution engine for coarse-grained data-parallel applications is proposed that allows developers to easily create large-scale distributed applications without requiring them to master concurrency techniques beyond being able to draw a graph of the data-dependencies of their algorithms. Based on the graph, a job manager intelligently distributes the work load so that system resources are used efficiently. The system is designed to scale from a small cluster of a few computers, or the multiple CPU cores on a powerful single computer, up to a data center containing thousands of servers.

Abstract translation: 提出了一种用于粗粒度数据并行应用程序的通用高性能分布式执行引擎，允许开发人员轻松创建大规模分布式应用程序，而不需要它们掌握并发技术，除了能够绘制数据依赖关系的图形 他们的算法。 基于该图，作业管理器智能地分配工作负载，以便有效地使用系统资源。 该系统旨在从几个计算机的小型集群或强大的单个计算机上的多个CPU核心扩展到包含数千台服务器的数据中心。

公开(公告)号：US20100241828A1

公开(公告)日：2010-09-23

申请号：US12406842

申请日：2009-03-18

Applicant: Yuan Yu ,  Pradeep Kumar Gunda ,  Michael A. Isard

Inventor： Yuan Yu ,  Pradeep Kumar Gunda ,  Michael A. Isard

IPC: G06F9/46 ,  G06F15/76 ,  G06F9/38

CPC classification number: H04L12/44 ,  G06F8/456 ,  G06F9/5066

Abstract: General-purpose distributed data-parallel computing using high-level computing languages is described. Data parallel portions of a sequential program written in a high-level language are automatically translated into a distributed execution plan. Map and reduction computations are automatically added to the plan. Patterns in the sequential program can be automatically identified to trigger map and reduction processing. Direct invocation of map and reduction processing is also provided. One or more portions of the reduce computation are pushed to the map stage and dynamic aggregation is inserted when possible. The system automatically identifies opportunities for partial reductions and aggregation, but also provides a set of extensions in a high-level computing language for the generation and optimization of the distributed execution plan. The extensions include annotations to declare functions suitable for these optimizations.

Abstract translation: 描述了使用高级计算语言的通用分布式数据并行计算。 以高级语言编写的顺序程序的数据并行部分将自动转换为分布式执行计划。 地图和缩小计算将自动添加到计划中。 顺序程序中的模式可以自动识别，以触发地图和缩小处理。 还提供了直接调用地图和缩小处理。 减少计算的一个或多个部分被推送到地图阶段，并且尽可能地插入动态聚合。 系统自动识别部分缩减和聚合的机会，但也提供了一组用于生成和优化分布式执行计划的高级计算语言的扩展。 扩展包括用于声明适合这些优化的函数的注释。

公开(公告)号：US20100235406A1

公开(公告)日：2010-09-16

申请号：US12404358

申请日：2009-03-16

Applicant: Oliver M.C. Williams ,  Michael A. Isard

Inventor： Oliver M.C. Williams ,  Michael A. Isard

IPC: G06F7/00 ,  G06F17/30 ,  G06Q99/00 ,  G06K9/46

CPC classification number: G06F17/30011 ,  G06Q10/06

Abstract: An image may be received, a portion of which corresponds to a surface of an object, such as a book, a CD, a DVD, a wine bottle, etc. The portion of the image that corresponds to the surface of the object is located. The portion of the image is compared with previously stored images of surfaces of objects to identify the object. A record of the object is created and added to a library. The record of the object may comprise the image of the object, the portion of the image which corresponds to the surface of the object, and/or the received image itself. The record may comprise an indicator of a location of the object.

Abstract translation: 可以接收图像，其一部分对应于诸如书籍，CD，DVD，酒瓶等的物体的表面。对应于物体表面的图像的部分位于 。 将图像的部分与先前存储的对象的表面的图像进行比较以识别对象。 创建对象的记录并将其添加到库中。 对象的记录可以包括对象的图像，对应于对象的表面的图像的部分和/或接收到的图像本身。 记录可以包括对象的位置的指示符。

公开(公告)号：US20090210457A1

公开(公告)日：2009-08-20

申请号：US12033715

申请日：2008-02-19

Applicant: Martin Abadi ,  Andrew D. Birrell ,  Timothy L. Harris ,  Michael A. Isard ,  Johnson Hsieh

Inventor： Martin Abadi ,  Andrew D. Birrell ,  Timothy L. Harris ,  Michael A. Isard ,  Johnson Hsieh

IPC: G06F12/00

CPC classification number: G06F9/466 ,  G06F8/443

Abstract: Strong semantics are provided to programs that are correctly synchronized in their use of transactions by using dynamic separation of objects that are accessed in transactions from those accessed outside transactions. At run-time, operations are performed to identify transitions between these protected and unprotected modes of access. Dynamic separation permits a range of hardware-based and software-based implementations which allow non-conflicting transactions to execute and commit in parallel. A run-time checking tool, analogous to a data-race detector, may be provided to test dynamic separation of transacted data and non-transacted data. Dynamic separation may be used in an asynchronous I/O library.