公开(公告)号：US08819106B1

公开(公告)日：2014-08-26

申请号：US12415649

申请日：2009-03-31

申请人： Peter Sirota ,  Richendra Khanna ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

发明人： Peter Sirota ,  Richendra Khanna ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

IPC分类号： G06F15/16

CPC分类号： H04L67/1008 ,  G06F9/485 ,  H04L29/08135 ,  H04L67/16

摘要： Techniques are described for managing distributed execution of programs. In some situations, the techniques include determining configuration information to be used for executing a particular program in a distributed manner on multiple computing nodes and/or include providing information and associated controls to a user regarding ongoing distributed execution of one or more programs to enable the user to modify the ongoing distributed execution in various manners. Determined configuration information may include, for example, configuration parameters such as a quantity of computing nodes and/or other measures of computing resources to be used for the executing, and may be determined in various manners, including by interactively gathering values for at least some types of configuration information from an associated user (e.g., via a GUI that is displayed to the user) and/or by automatically determining values for at least some types of configuration information (e.g., for use as recommendations to a user).

摘要翻译： 描述了用于管理程序的分布式执行的技术。 在某些情况下，技术包括确定用于在多个计算节点上以分布式方式执行特定程序的配置信息和/或包括向用户提供关于一个或多个程序的正在进行的分布式执行的信息和相关控制，以使能 用户以各种方式修改正在进行的分布式执行。 确定的配置信息可以包括例如诸如计算节点的数量的配置参数和/或要用于执行的计算资源的其他度量，并且可以以各种方式来确定，包括通过交互地收集至少一些 来自相关用户的配置信息的类型（例如，经由显示给用户的GUI）和/或通过自动确定至少一些类型的配置信息（例如，用作对用户的建议）的值。

公开(公告)号：US09329909B1

公开(公告)日：2016-05-03

申请号：US13620805

申请日：2012-09-15

申请人： Richendra Khanna ,  Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

发明人： Richendra Khanna ,  Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

IPC分类号： G06F15/16 ,  G06F9/50

CPC分类号： H04L67/1029 ,  G06F9/5072 ,  G06F9/5083

摘要： Techniques are described for managing distributed execution of programs. In some situations, the techniques include dynamically modifying the distributed program execution in various manners, such as based on monitored status information. The dynamic modifying of the distributed program execution may include adding and/or removing computing nodes from a cluster that is executing the program, modifying the amount of computing resources that are available for the distributed program execution, terminating or temporarily suspending execution of the program (e.g., if an insufficient quantity of computing nodes of the cluster are available to perform execution), etc.

摘要翻译： 描述了用于管理程序的分布式执行的技术。 在某些情况下，这些技术包括以各种方式动态地修改分布式程序执行，例如基于被监视的状态信息。 分布式程序执行的动态修改可以包括从执行程序的集群中添加和/或移除计算节点，修改可用于分布式程序执行的计算资源的数量，终止或暂时中止程序的执行（ 例如，如果集群的计算节点数量不足可用于执行）等等

公开(公告)号：US08321558B1

公开(公告)日：2012-11-27

申请号：US12415773

申请日：2009-03-31

申请人： Peter Sirota ,  Richendra Khanna ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

发明人： Peter Sirota ,  Richendra Khanna ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

IPC分类号： G06F15/16

CPC分类号： G06F9/5011 ,  G06F2209/504 ,  Y02D10/22

摘要： Techniques are described for managing distributed execution of programs. In some situations, the techniques include dynamically monitoring the ongoing distributed execution of a program on a cluster of multiple computing nodes, and may include automatically determining the status of execution of the program on each of the multiple computing nodes and/or automatically determining the aggregate usage of one or more types of computing resources across the cluster of multiple computing nodes by the distributed program execution. The information obtained from the dynamic monitoring may be used in various manners, including to facilitate dynamically modifying the ongoing distributed program execution in various manners, such as to temporarily throttle usage of computing resources by the distributed program execution (e.g., to remove or reduce one or more bottlenecks).

摘要翻译： 描述了用于管理程序的分布式执行的技术。 在一些情况下，这些技术包括动态地监视多个计算节点的集群上的程序的正在进行的分布式执行，并且可以包括在多个计算节点的每一个上自动确定该程序的执行状态和/或自动地确定该集合 通过分布式程序执行在多个计算节点的群集之间使用一种或多种类型的计算资源。 从动态监视获得的信息可以以各种方式使用，包括有助于以各种方式动态地修改正在进行的分布式程序执行，例如通过分布式程序执行临时调节计算资源的使用（例如，去除或减少一个 或更多的瓶颈）。

公开(公告)号：US08296419B1

公开(公告)日：2012-10-23

申请号：US12415725

申请日：2009-03-31

申请人： Richendra Khanna ,  Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

发明人： Richendra Khanna ,  Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

IPC分类号： G06F15/16

CPC分类号： H04L67/1029 ,  G06F9/5072 ,  G06F9/5083

摘要： Techniques are described for managing distributed execution of programs. In some situations, the techniques include dynamically modifying the distributed program execution in various manners, such as based on monitored status information. The dynamic modifying of the distributed program execution may include adding and/or removing computing nodes from a cluster that is executing the program, modifying the amount of computing resources that are available for the distributed program execution, terminating or temporarily suspending execution of the program (e.g., if an insufficient quantity of computing nodes of the cluster are available to perform execution), etc.

摘要翻译： 描述了用于管理程序的分布式执行的技术。 在某些情况下，这些技术包括以各种方式动态地修改分布式程序执行，例如基于被监视的状态信息。 分布式程序执行的动态修改可以包括从执行程序的集群中添加和/或移除计算节点，修改可用于分布式程序执行的计算资源的数量，终止或暂时中止程序的执行（ 例如，如果集群的计算节点数量不足可用于执行）等等

公开(公告)号：US08370493B2

公开(公告)日：2013-02-05

申请号：US12334004

申请日：2008-12-12

申请人： Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Richendra Khanna ,  Luis Felipe Cabrera

发明人： Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Richendra Khanna ,  Luis Felipe Cabrera

IPC分类号： G06F15/173

CPC分类号： G06F11/1451 ,  G06F9/4806 ,  G06F9/485 ,  G06F11/1469

摘要： Techniques are described for managing distributed execution of programs. In at least some situations, the techniques include decomposing or otherwise separating the execution of a program into multiple distinct execution jobs that may each be executed on a distinct computing node, such as in a parallel manner with each execution job using a distinct subset of input data for the program. In addition, the techniques may include temporarily terminating and later resuming execution of at least some execution jobs, such as by persistently storing an intermediate state of the partial execution of an execution job, and later retrieving and using the stored intermediate state to resume execution of the execution job from the intermediate state. Furthermore, the techniques may be used in conjunction with a distributed program execution service that executes multiple programs on behalf of multiple customers or other users of the service.

公开(公告)号：US20100153955A1

公开(公告)日：2010-06-17

申请号：US12334004

申请日：2008-12-12

申请人： Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Richendra Khanna ,  Luis Felipe Cabrera

发明人： Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Richendra Khanna ,  Luis Felipe Cabrera

IPC分类号： G06F9/46

CPC分类号： G06F11/1451 ,  G06F9/4806 ,  G06F9/485 ,  G06F11/1469

摘要： Techniques are described for managing distributed execution of programs. In at least some situations, the techniques include decomposing or otherwise separating the execution of a program into multiple distinct execution jobs that may each be executed on a distinct computing node, such as in a parallel manner with each execution job using a distinct subset of input data for the program. In addition, the techniques may include temporarily terminating and later resuming execution of at least some execution jobs, such as by persistently storing an intermediate state of the partial execution of an execution job, and later retrieving and using the stored intermediate state to resume execution of the execution job from the intermediate state. Furthermore, the techniques may be used in conjunction with a distributed program execution service that executes multiple programs on behalf of multiple customers or other users of the service.

摘要翻译： 描述了用于管理程序的分布式执行的技术。 在至少一些情况下，这些技术包括将程序的执行分解或以其他方式分离成可以在不同的计算节点上执行的多个不同的执行作业，例如以每个执行作业使用不同的输入子集的并行方式 程序数据。 此外，这些技术可以包括临时终止和稍后恢复至少一些执行作业的执行，例如通过持续地存储执行作业的部分执行的中间状态，以及稍后检索和使用存储的中间状态来恢复执行 执行作业从中间状态。 此外，这些技术可以与代表多个客户或服务的其他用户执行多个程序的分布式程序执行服务结合使用。

公开(公告)号：US08260840B1

公开(公告)日：2012-09-04

申请号：US12825073

申请日：2010-06-28

申请人： Peter Sirota ,  Richard J. Cole ,  Vaibhav Aggarwal ,  Andrew J. Hitchcock ,  Jai Vasanth

发明人： Peter Sirota ,  Richard J. Cole ,  Vaibhav Aggarwal ,  Andrew J. Hitchcock ,  Jai Vasanth

IPC分类号： G06F15/16

CPC分类号： G06F9/5061

摘要： Techniques are described for managing distributed execution of programs, including by dynamically scaling a cluster of multiple computing nodes used to perform ongoing distributed execution of a program, such as to increase and/or decrease the quantity of computing nodes in the cluster at various times and for various reasons. An architecture may be used that facilitates the dynamic scaling of a cluster, including by having at least some of the computing nodes act as core nodes that each participate in a distributed storage system for the distributed program execution, and having one or more other computing nodes that act as auxiliary nodes that do not participate in the distributed storage system. If computing nodes are selected to be removed from the cluster during ongoing distributed execution of a program, one or more nodes of the auxiliary computing node type may be selected for the removal.

摘要翻译： 描述了用于管理程序的分布式执行的技术，包括通过动态地缩放用于执行程序的持续分布式执行的多个计算节点的集群，例如在不同时间增加和/或减少集群中的计算节点的数量， 由于种种原因 可以使用有助于群集的动态缩放的架构，包括通过使至少一些计算节点充当每个参与用于分布式程序执行的分布式存储系统的核心节点，并且具有一个或多个其他计算节点 作为不参与分布式存储系统的辅助节点。 如果在程序的分布式执行期间选择计算节点从集群中删除，则可以选择辅助计算节点类型的一个或多个节点用于删除。

公开(公告)号：US08719415B1

公开(公告)日：2014-05-06

申请号：US12825097

申请日：2010-06-28

申请人： Peter Sirota ,  Richard J. Cole ,  Vaibhav Aggarwal ,  Andrew J. Hitchcock ,  Jai Vasanth

发明人： Peter Sirota ,  Richard J. Cole ,  Vaibhav Aggarwal ,  Andrew J. Hitchcock ,  Jai Vasanth

IPC分类号： G06F15/173 ,  G06F15/177

CPC分类号： G06F9/4881 ,  G06F9/5011 ,  G06F9/5061 ,  G06F2209/5011 ,  H04L67/10

摘要： Techniques are described for managing distributed execution of programs, including by dynamically scaling a cluster of multiple computing nodes performing ongoing distributed execution of a program, such as to increase and/or decrease computing node quantity. An architecture may be used that has core nodes that each participate in a distributed storage system for the distributed program execution, and that has one or more other auxiliary nodes that do not participate in the distributed storage system. Furthermore, as part of performing the dynamic scaling of a cluster, computing nodes that are only temporarily available may be selected and used, such as computing nodes that might be removed from the cluster during the ongoing program execution to be put to other uses and that may also be available for a different fee (e.g., a lower fee) than other computing nodes that are available throughout the ongoing use of the cluster.

摘要翻译： 描述了用于管理程序的分布式执行的技术，包括通过动态地缩放执行程序的正在进行的分布式执行的多个计算节点的集群，例如增加和/或减少计算节点数量。 可以使用具有每个参与用于分布式程序执行的分布式存储系统的核心节点并且具有不参与分布式存储系统的一个或多个其他辅助节点的架构。 此外，作为执行群集的动态缩放的一部分，可以选择和使用仅临时可用的计算节点，例如在正在进行的程序执行期间可能从群集中移除的计算节点以被放置到其他用途， 也可能与在整个持续使用集群时可用的其他计算节点的费用不同（例如，较低的费用）可用。

公开(公告)号：US08418181B1

公开(公告)日：2013-04-09

申请号：US12476561

申请日：2009-06-02

申请人： Peter Sirota ,  Richendra Khanna

发明人： Peter Sirota ,  Richendra Khanna

IPC分类号： G06F9/46 ,  G06F15/16 ,  G06F15/173

CPC分类号： H04L67/00 ,  G06F9/5033

摘要： Techniques are described for managing execution of programs, such as for distributed execution of a program on multiple computing nodes. In some situations, the techniques include selecting a cluster of computing nodes to use for executing a program based at least in part on data to be used during the program execution. For example, the computing node selection for a particular program may be performed so as to attempt to identify and use computing nodes that already locally store some or all of the input data that will be used by those computing nodes as part of the executing of that program on those nodes. Such techniques may provide benefits in a variety of situations, including when the size of input datasets to be used by a program are large, and the transferring of data to and/or from computing nodes may impose large delays and/or monetary costs.

摘要翻译： 描述了用于管理程序的执行的技术，例如用于在多个计算节点上的程序的分布式执行。 在一些情况下，这些技术包括至少部分地基于要在程序执行期间使用的数据来选择用于执行程序的计算节点的集群。 例如，可以执行特定程序的计算节点选择，以便尝试识别和使用已经本地存储将被这些计算节点使用的一些或全部输入数据的计算节点，作为该执行的一部分 程序在这些节点上。 这种技术可以在各种情况下提供益处，包括当程序使用的输入数据集的大小较大时，以及向计算节点和/或从计算节点传输数据可能会施加大的延迟和/或货币成本。

公开(公告)号：US08856291B2

公开(公告)日：2014-10-07

申请号：US13396479

申请日：2012-02-14

申请人： James P. Bartlett ,  Richard J. Cole ,  Adam D. Gray ,  Peter Sirota

发明人： James P. Bartlett ,  Richard J. Cole ,  Adam D. Gray ,  Peter Sirota

IPC分类号： G06F15/177

CPC分类号： G06F9/5027 ,  G06Q10/0633 ,  G06Q10/103 ,  H04L41/0803

摘要： Techniques are described for providing clients with access to functionality for creating, configuring and executing defined workflows that manipulate source data in defined manners, such as under the control of a configurable workflow service that is available to multiple remote clients over one or more public networks. A defined workflow for a client may, for example, include multiple interconnected workflow components that are specified by the client and that each are configured to perform one or more types of data manipulation operations on a specified type of input data. The configurable workflow service may further execute the defined workflow at one or more times and in one or more manners, such as in some situations by provisioning multiple computing nodes provided by the configurable workflow service to each implement at least one of the workflow components for the defined workflow.

摘要翻译： 描述了技术，用于为客户提供访问功能的功能，以创建，配置和执行以定义的方式操纵源数据的定义的工作流，例如在可配置工作流服务的控制下，多个远程客户端可通过一个或多个公共网络使用。 客户端定义的工作流可以例如包括由客户端指定的多个互连的工作流组件，并且每个被配置为对指定类型的输入数据执行一种或多种类型的数据操纵操作。 可配置工作流服务可以在一个或多个时间并且以一种或多种方式进一步执行所定义的工作流，诸如在某些情况下，通过将由可配置工作流服务提供的多个计算节点提供给每个实施方案来实现至少一个工作流组件 定义的工作流程。