公开(公告)号：US20140330981A1

公开(公告)日：2014-11-06

申请号：US14338150

申请日：2014-07-22

Applicant: Amazon Technologies, Inc.

Inventor： Peter Sirota ,  Richendra Khanna ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

IPC: G06F15/173

CPC classification number: H04L67/1008 ,  G06F9/485 ,  H04L29/08135 ,  H04L67/16

Abstract: Techniques are described for managing distributed execution of programs. In some situations, the techniques include determining configuration information to be used for executing a particular program in a distributed manner on multiple computing nodes and/or include providing information and associated controls to a user regarding ongoing distributed execution of one or more programs to enable the user to modify the ongoing distributed execution in various manners. Determined configuration information may include, for example, configuration parameters such as a quantity of computing nodes and/or other measures of computing resources to be used for the executing, and may be determined in various manners, including by interactively gathering values for at least some types of configuration information from an associated user (e.g., via a GUI that is displayed to the user) and/or by automatically determining values for at least some types of configuration information (e.g., for use as recommendations to a user).

Abstract translation: 描述了用于管理程序的分布式执行的技术。 在某些情况下，技术包括确定用于在多个计算节点上以分布式方式执行特定程序的配置信息和/或包括向用户提供关于一个或多个程序的正在进行的分布式执行的信息和相关控制，以使能 用户以各种方式修改正在进行的分布式执行。 确定的配置信息可以包括例如诸如计算节点的数量的配置参数和/或要用于执行的计算资源的其他度量，并且可以以各种方式来确定，包括通过交互地收集至少一些 来自相关用户的配置信息的类型（例如，经由显示给用户的GUI）和/或通过自动确定至少一些类型的配置信息（例如，用作对用户的建议）的值。

公开(公告)号：US10324761B2

公开(公告)日：2019-06-18

申请号：US14918418

申请日：2015-10-20

Applicant: Amazon Technologies, Inc.

Inventor： James P. Bartlett ,  Richard J. Cole ,  Adam D. Gray ,  Peter Sirota

IPC: G06F9/50 ,  G06Q10/06 ,  G06Q10/10 ,  H04L12/24

Abstract: Techniques are described for providing clients with access to functionality for creating, configuring and executing defined workflows that manipulate source data in defined manners, such as under the control of a configurable workflow service that is available to multiple remote clients over one or more public networks. A defined workflow for a client may, for example, include multiple interconnected workflow components that are specified by the client and that each are configured to perform one or more types of data manipulation operations on a specified type of input data. The configurable workflow service may further execute the defined workflow at one or more times and in one or more manners, such as in some situations by provisioning multiple computing nodes provided by the configurable workflow service to each implement at least one of the workflow components for the defined workflow.

公开(公告)号：US09898522B2

公开(公告)日：2018-02-20

申请号：US14572562

申请日：2014-12-16

Applicant: Amazon Technologies, Inc.

Inventor： Richard J. Cole ,  Alan D. Mock

IPC: G06F17/30

CPC classification number: G06F17/30592 ,  G06F17/3033 ,  G06F17/30489

Abstract: Techniques are described for managing aggregation of data in a distributed manner, such as for a particular client based on specified configuration information. The described techniques may include storing aggregated data values for an OLAP cube or other data structure in a distributed manner, such as in some situations in a distributed hash table. The aggregated data values to be stored may be generated in various manners, such as by performing multi-stage data manipulation operations—for example, a map-reduce architecture may be used, with a first stage involving the use of one or more specified map functions to be performed, and with at least a second stage involving the use of one or more specified reduce functions to be performed.

公开(公告)号：US20160234300A1

公开(公告)日：2016-08-11

申请号：US15133098

申请日：2016-04-19

Applicant: Amazon Technologies, Inc.

Inventor： Richendra Khanna ,  Peter Sirota ,  Ian P. Nowland ,  Richard J. Cole ,  Jai Vasanth ,  Andrew J. Hitchcock ,  Luis Felipe Cabrera

IPC: H04L29/08

CPC classification number: H04L67/1029 ,  G06F9/5072 ,  G06F9/5083

Abstract: Techniques are described for managing distributed execution of programs. In some situations, the techniques include dynamically modifying the distributed program execution in various manners, such as based on monitored status information. The dynamic modifying of the distributed program execution may include adding and/or removing computing nodes from a cluster that is executing the program, modifying the amount of computing resources that are available for the distributed program execution, terminating or temporarily suspending execution of the program (e.g., if an insufficient quantity of computing nodes of the cluster are available to perform execution), etc.

Abstract translation: 描述了用于管理程序的分布式执行的技术。 在某些情况下，这些技术包括以各种方式动态地修改分布式程序执行，例如基于被监视的状态信息。 分布式程序执行的动态修改可以包括从执行程序的集群中添加和/或移除计算节点，修改可用于分布式程序执行的计算资源的数量，终止或暂时中止程序的执行（ 例如，如果集群的计算节点数量不足可用于执行）等等

公开(公告)号：US20160041846A1

公开(公告)日：2016-02-11

申请号：US14918418

申请日：2015-10-20

Applicant: Amazon Technologies, Inc.

Inventor： James P. Bartlett ,  Richard J. Cole ,  Adam D. Gray ,  Peter Sirota

IPC: G06F9/50

CPC classification number: G06F9/5027 ,  G06Q10/0633 ,  G06Q10/103 ,  H04L41/0803

Abstract: Techniques are described for providing clients with access to functionality for creating, configuring and executing defined workflows that manipulate source data in defined manners, such as under the control of a configurable workflow service that is available to multiple remote clients over one or more public networks. A defined workflow for a client may, for example, include multiple interconnected workflow components that are specified by the client and that each are configured to perform one or more types of data manipulation operations on a specified type of input data. The configurable workflow service may further execute the defined workflow at one or more times and in one or more manners, such as in some situations by provisioning multiple computing nodes provided by the configurable workflow service to each implement at least one of the workflow components for the defined workflow.

公开(公告)号：US20150135185A1

公开(公告)日：2015-05-14

申请号：US14598137

申请日：2015-01-15

Applicant: Amazon Technologies, Inc.

Inventor： Peter Sirota ,  Richard J. Cole ,  Vaibhav Aggarwal ,  Andrew J. Hitchcock ,  Jai Vasanth

IPC: G06F9/48 ,  G06F9/50

CPC classification number: G06F9/4881 ,  G06F9/5011 ,  G06F9/5061 ,  G06F2209/5011 ,  H04L67/10

Abstract: Techniques are described for managing distributed execution of programs, including by dynamically scaling a cluster of multiple computing nodes performing ongoing distributed execution of a program, such as to increase and/or decrease computing node quantity. An architecture may be used that has core nodes that each participate in a distributed storage system for the distributed program execution, and that has one or more other auxiliary nodes that do not participate in the distributed storage system. Furthermore, as part of performing the dynamic scaling of a cluster, computing nodes that are only temporarily available may be selected and used, such as computing nodes that might be removed from the cluster during the ongoing program execution to be put to other uses and that may also be available for a different fee (e.g., a lower fee) than other computing nodes that are available throughout the ongoing use of the cluster.

Abstract translation: 描述了用于管理程序的分布式执行的技术，包括通过动态地缩放执行程序的正在进行的分布式执行的多个计算节点的集群，例如增加和/或减少计算节点数量。 可以使用具有每个参与用于分布式程序执行的分布式存储系统的核心节点并且具有不参与分布式存储系统的一个或多个其他辅助节点的架构。 此外，作为执行群集的动态缩放的一部分，可以选择和使用仅临时可用的计算节点，例如在正在进行的程序执行期间可能从群集中移除的计算节点以被放置到其他用途， 也可能与在整个持续使用集群时可用的其他计算节点的费用不同（例如，较低的费用）可用。

公开(公告)号：US20150106325A1

公开(公告)日：2015-04-16

申请号：US14572562

申请日：2014-12-16

Applicant: Amazon Technologies, Inc.

Inventor： Richard J. Cole ,  Alan D. Mock

IPC: G06F17/30

CPC classification number: G06F17/30592 ,  G06F17/3033 ,  G06F17/30489

Abstract: Techniques are described for managing aggregation of data in a distributed manner, such as for a particular client based on specified configuration information. The described techniques may include storing aggregated data values for an OLAP cube or other data structure in a distributed manner, such as in some situations in a distributed hash table. The aggregated data values to be stored may be generated in various manners, such as by performing multi-stage data manipulation operations—for example, a map-reduce architecture may be used, with a first stage involving the use of one or more specified map functions to be performed, and with at least a second stage involving the use of one or more specified reduce functions to be performed.

Abstract translation: 描述了以分布式方式管理数据的聚合的技术，例如基于指定配置信息对于特定客户端的聚合。 所描述的技术可以包括以分布式方式存储用于OLAP多维数据集或其他数据结构的聚合数据值，例如在分布式哈希表中的某些情况下。 要存储的聚合数据值可以以各种方式生成，例如通过执行多级数据操作操作 - 例如，可以使用地图缩小架构，其中第一阶段涉及使用一个或多个指定的地图 功能，并且至少涉及使用一个或多个指定的减少功能执行的第二阶段。

公开(公告)号：US09280390B2

公开(公告)日：2016-03-08

申请号：US14598137

申请日：2015-01-15

Applicant: Amazon Technologies, Inc.

Inventor： Peter Sirota ,  Richard J. Cole ,  Vaibhav Aggarwal ,  Andrew J. Hitchcock ,  Jai Vasanth

IPC: G06F15/177 ,  H04L12/28 ,  G06F9/48 ,  G06F9/50 ,  H04L29/08

CPC classification number: G06F9/4881 ,  G06F9/5011 ,  G06F9/5061 ,  G06F2209/5011 ,  H04L67/10

Abstract: Techniques are described for managing distributed execution of programs, including by dynamically scaling a cluster of multiple computing nodes performing ongoing distributed execution of a program, such as to increase and/or decrease computing node quantity. An architecture may be used that has core nodes that each participate in a distributed storage system for the distributed program execution, and that has one or more other auxiliary nodes that do not participate in the distributed storage system. Furthermore, as part of performing the dynamic scaling of a cluster, computing nodes that are only temporarily available may be selected and used, such as computing nodes that might be removed from the cluster during the ongoing program execution to be put to other uses and that may also be available for a different fee (e.g., a lower fee) than other computing nodes that are available throughout the ongoing use of the cluster.

Abstract translation: 描述了用于管理程序的分布式执行的技术，包括通过动态地缩放执行程序的正在进行的分布式执行的多个计算节点的集群，例如增加和/或减少计算节点数量。 可以使用具有每个参与用于分布式程序执行的分布式存储系统的核心节点并且具有不参与分布式存储系统的一个或多个其他辅助节点的架构。 此外，作为执行群集的动态缩放的一部分，可以选择和使用仅临时可用的计算节点，例如在正在进行的程序执行期间可能从群集中移除的计算节点以被放置到其他用途， 也可能与在整个持续使用集群时可用的其他计算节点的费用不同（例如，较低的费用）可用。

公开(公告)号：US09184988B2

公开(公告)日：2015-11-10

申请号：US14481553

申请日：2014-09-09

Applicant: Amazon Technologies, Inc.

Inventor： James P. Bartlett ,  Richard J. Cole ,  Adam D. Gray ,  Peter Sirota

IPC: H04L12/24 ,  G06Q10/06 ,  G06Q10/10

CPC classification number: G06F9/5027 ,  G06Q10/0633 ,  G06Q10/103 ,  H04L41/0803

Abstract: Techniques are described for providing clients with access to functionality for creating, configuring and executing defined workflows that manipulate source data in defined manners, such as under the control of a configurable workflow service that is available to multiple remote clients over one or more public networks. A defined workflow for a client may, for example, include multiple interconnected workflow components that are specified by the client and that each are configured to perform one or more types of data manipulation operations on a specified type of input data. The configurable workflow service may further execute the defined workflow at one or more times and in one or more manners, such as in some situations by provisioning multiple computing nodes provided by the configurable workflow service to each implement at least one of the workflow components for the defined workflow.

Abstract translation: 描述了技术，用于为客户提供访问功能的功能，以创建，配置和执行以定义的方式操纵源数据的定义的工作流，例如在可配置工作流服务的控制下，多个远程客户端可通过一个或多个公共网络使用。 客户端定义的工作流可以例如包括由客户端指定的多个互连的工作流组件，并且每个被配置为对指定类型的输入数据执行一种或多种类型的数据操纵操作。 可配置工作流服务可以在一个或多个时间并且以一种或多种方式进一步执行所定义的工作流，诸如在某些情况下，通过将由可配置工作流服务提供的多个计算节点提供给每个实施方案来实现至少一个工作流组件 定义的工作流程。

公开(公告)号：US08966030B1

公开(公告)日：2015-02-24

申请号：US14231435

申请日：2014-03-31

Applicant: Amazon Technologies, Inc.

Inventor： Peter Sirota ,  Richard J. Cole ,  Vaibhav Aggarwal ,  Andrew J. Hitchcock ,  Jai Vasanth

IPC: G06F15/177 ,  H04L29/08

CPC classification number: G06F9/4881 ,  G06F9/5011 ,  G06F9/5061 ,  G06F2209/5011 ,  H04L67/10

Abstract: Techniques are described for managing distributed execution of programs, including by dynamically scaling a cluster of multiple computing nodes performing ongoing distributed execution of a program, such as to increase and/or decrease computing node quantity. An architecture may be used that has core nodes that each participate in a distributed storage system for the distributed program execution, and that has one or more other auxiliary nodes that do not participate in the distributed storage system. Furthermore, as part of performing the dynamic scaling of a cluster, computing nodes that are only temporarily available may be selected and used, such as computing nodes that might be removed from the cluster during the ongoing program execution to be put to other uses and that may also be available for a different fee (e.g., a lower fee) than other computing nodes that are available throughout the ongoing use of the cluster.

Abstract translation: 描述了用于管理程序的分布式执行的技术，包括通过动态地缩放执行程序的正在进行的分布式执行的多个计算节点的集群，例如增加和/或减少计算节点数量。 可以使用具有每个参与用于分布式程序执行的分布式存储系统的核心节点并且具有不参与分布式存储系统的一个或多个其他辅助节点的架构。 此外，作为执行群集的动态缩放的一部分，可以选择和使用仅临时可用的计算节点，例如在正在进行的程序执行期间可能从群集中移除的计算节点以被放置到其他用途， 也可能与在整个持续使用集群时可用的其他计算节点的费用不同（例如，较低的费用）可用。