Automatic exploitation of data parallelism in streaming applications
    4.
    发明授权
    Automatic exploitation of data parallelism in streaming applications 有权
    在流媒体应用中自动利用数据并行性

    公开(公告)号:US09170794B2

    公开(公告)日:2015-10-27

    申请号:US13596676

    申请日:2012-08-28

    IPC分类号: G06F9/45

    CPC分类号: G06F8/456

    摘要: An embodiment of the invention provides a method for exploiting stateless and stateful data parallelism in a streaming application, wherein a compiler determines whether an operator of the streaming application is safe to parallelize based on a definition of the operator and an instance of the definition. The operator is not safe to parallelize when the operator has selectivity greater than 1, wherein the selectivity is the number of output tuples generated for each input tuple. Parallel regions are formed within the streaming application with the compiler when the operator is safe to parallelize. Synchronization strategies for the parallel regions are determined with the compiler, wherein the synchronization strategies are determined based on the definition of the operator and the instance of the definition. The synchronization strategies of the parallel regions are enforced with a runtime system.

    摘要翻译: 本发明的实施例提供了一种在流应用中利用无状态和有状态数据并行性的方法,其中编译器基于操作者的定义和定义的实例来确定流应用的操作者是否安全地进行并行化。 当操作员的选择性大于1时,操作员不能并行化,其中选择性是为每个输入元组生成的输出元组的数量。 当操作员安全地并行化时,并行区域与编译器在流应用程序内形成。 使用编译器确定并行区域的同步策略,其中基于运算符的定义和定义的实例来确定同步策略。 并行区域的同步策略是通过运行时系统实现的。

    Application resource model composition from constituent components
    5.
    发明授权
    Application resource model composition from constituent components 有权
    组成部分的应用资源模型组成

    公开(公告)号:US09135069B2

    公开(公告)日:2015-09-15

    申请号:US13464386

    申请日:2012-05-04

    CPC分类号: G06F9/5011 H04L41/5035

    摘要: Techniques for composing an application resource model are disclosed. The techniques include obtaining operator-level metrics from an execution of a data stream processing application according to a first configuration, wherein the application is executed by nodes of the data stream processing system and the application includes processing elements comprised of multiple operators, wherein two or more of the operators are combined in a first combination to form a processing element according to the first configuration, generating operator-level resource functions from the first combination of operators based on the obtained operator-level metrics, and generating a processing element-level resource function using the generated operator-level resource functions to predict a model for the processing element formed by a second combination of operators, the processing element-level resource function representing an application resource model usable for predicting characteristics of the application executed according to a second configuration.

    摘要翻译: 公开了组合应用资源模型的技术。 这些技术包括根据第一配置从数据流处理应用的执行获得运营商级度量,其中应用由数据流处理系统的节点执行,并且应用包括由多个运营商组成的处理元件,其中两个或 更多的运营商以第一组合组合以形成根据第一配置的处理元件,基于所获得的运营商级度量从运营商的第一组合生成运营商级资源功能,以及生成处理元件级资源 使用所生成的操作者级资源函数来预测由第二组合运算符形成的处理元素的模型,所述处理元素级资源函数表示可用于根据第二配置来执行的用于预测所述应用的特征的应用资源模型 。

    Method for high-performance data stream processing
    6.
    发明授权
    Method for high-performance data stream processing 有权
    高性能数据流处理方法

    公开(公告)号:US08949810B2

    公开(公告)日:2015-02-03

    申请号:US12139651

    申请日:2008-06-16

    IPC分类号: G06F9/45

    CPC分类号: G06F8/451

    摘要: Techniques for optimizing data stream processing are provided. The techniques include employing a pattern, wherein the pattern facilitates splitting of one or more incoming streams and distributing processing across one or more operators, obtaining one or mote operators, wherein the one or more operators support at least one group-independent aggregation and join operation on one or more streams, generating code, wherein the code facilitates mapping of the application onto a computational infrastructure to enable workload partitioning, using the one or more operators to decompose each of the application into one or more granular components, and using the code to reassemble the one or more granular components into one or more deployable blocks to map the application to a computational infrastructure, wherein reassembling the one or more granular components to map the application to the computational infrastructure optimizes data stream processing of the application.

    摘要翻译: 提供了优化数据流处理的技术。 所述技术包括采用模式,其中所述模式有助于分离一个或多个输入流并且跨越一个或多个运算符分配处理,获得一个或多个运算符,其中所述一个或多个运算符支持至少一个独立于组的聚合和连接操作 在一个或多个流上生成代码,其中所述代码有助于将应用程序映射到计算基础设施上以实现工作负载划分,使用所述一个或多个运算符将每个应用程序分解为一个或多个粒度组件,并使用代码 将所述一个或多个粒状部件重组为一个或多个可部署块以将所述应用映射到计算基础设施,其中重新组装所述一个或多个粒度组件以将所述应用映射到所述计算基础设施来优化所述应用的数据流处理。

    Determining operator partitioning constraint feasibility
    7.
    发明授权
    Determining operator partitioning constraint feasibility 有权
    确定运算符分区约束的可行性

    公开(公告)号:US08806510B2

    公开(公告)日:2014-08-12

    申请号:US12556931

    申请日:2009-09-10

    CPC分类号: G06F9/5066

    摘要: Techniques for determining feasibility of a set of one or more operator partitioning constraints are provided. The techniques include receiving one or more sets of operator partitioning constraints, wherein each set of one or more constraints define one or more desired conditions for grouping together of operators into partitions and placing partitions on hosts, wherein each operator is embodied as software that performs a particular function, processing each set of one or more operator partitioning constraints to determine feasibility of each set of one or more operator partitioning constraints, creating and outputting one or more candidate partitions and one or more host placements for each set of feasible partitioning constraints, and creating and outputting a certificate of infeasibility for each set of infeasible partitioning constraints, wherein the certificate of infeasibility outlines one or more reasons for infeasibility.

    摘要翻译: 提供了用于确定一组或多个操作者分区约束的可行性的技术。 这些技术包括接收一组或多组运算符分配约束,其中每组一个或多个约束定义一个或多个期望条件,用于将运算符分组到分区中并在主机上放置分区,其中每个运算符被实施为执行 特定功能,处理每组一个或多个运营商划分约束以确定每组一个或多个运营商分区约束的可行性,为每组可行分区约束创建和输出一个或多个候选分区和一个或多个主机布局,以及 创建和输出每组不可行分区约束的不可行性证书,其中不可行性证书概述了一个或多个不可行性的原因。

    STREAM PROCESSING WITH RUNTIME ADAPTATION
    8.
    发明申请
    STREAM PROCESSING WITH RUNTIME ADAPTATION 有权
    具有运行适应性的流程处理

    公开(公告)号:US20140059210A1

    公开(公告)日:2014-02-27

    申请号:US13594961

    申请日:2012-08-27

    IPC分类号: G06F15/173

    摘要: Embodiments of the disclosure include a system for providing stream processing with runtime adaptation, having a stream processing application that receives an incoming data stream and a runtime infrastructure configured to execute the stream processing application. The system also includes an orchestrator configured to communicate with the runtime infrastructure and the stream processing application, the orchestrator configured to perform a method. The method includes registering one or more events, wherein each of the events is associated with a stream processing application. The method also includes monitoring, by a processor, for an occurrence of the one or more events associated with the stream processing application, wherein each of the one or more events is associated with one or more runtime metrics. The method further includes receiving an event notification, wherein the event notification includes event identification and an event context and executing an adaptation of the stream processing application.

    摘要翻译: 本公开的实施例包括用于提供具有运行时适应的流处理的系统,具有接收输入数据流的流处理应用和被配置为执行流处理应用的运行时基础设施。 该系统还包括经配置以与运行时基础设施和流处理应用程序进行通信的协调者,该协调器被配置为执行方法。 该方法包括登记一个或多个事件,其中每个事件与流处理应用相关联。 该方法还包括由处理器监视与流处理应用相关联的一个或多个事件的发生,其中所述一个或多个事件中的每一个与一个或多个运行时度量相关联。 该方法还包括接收事件通知,其中事件通知包括事件识别和事件上下文以及执行流处理应用的适应。

    Selective processing of location-sensitive data streams

    公开(公告)号:US08295856B2

    公开(公告)日:2012-10-23

    申请号:US12575371

    申请日:2009-10-07

    IPC分类号: H04W24/00

    摘要: A method for processing a first data stream specifying locations of a user at different times and a second data stream specifying values of a monitored attribute at a location of interest at different times includes: receiving a location-centric trigger specifying at least one spatial predicate condition relative to the location of interest and at least one non-spatial predicate condition relevant to the location of interest, calculating a safe region that includes locations whose probability of satisfying the spatial predicate condition falls below a first threshold, calculating a safe value container that includes values whose probability of satisfying the non-spatial predicate condition falls below a second threshold, and processing the first data stream and the second data stream against the location-centric trigger, by considering only those locations that are not contained within the safe region and only those values that are not contained within the safe value container.

    Application resource model composition from constituent components
    10.
    发明授权
    Application resource model composition from constituent components 失效
    组成部分的应用资源模型组成

    公开(公告)号:US08255554B2

    公开(公告)日:2012-08-28

    申请号:US12466017

    申请日:2009-05-14

    IPC分类号: G06F15/16 G06F9/44

    CPC分类号: G06F9/5011 H04L41/5035

    摘要: Techniques for composing an application resource model in a data stream processing system are disclosed. The application resource model may be used to understand what resources will be consumed by an application when executed by the data stream processing system. For example, a method for composing an application resource model for a data stream processing system comprises the following steps. One or more operator-level metrics are obtained from an execution of a data stream processing application in accordance with a first configuration. The application is executed by one or more nodes of the data stream processing system, and the application is comprised of one or more processing elements that are comprised of one or more operators. One or more operator-level resource functions are generated based on the obtained one or more operator-level metrics. A processing element-level resource function is generated based on the one or more generated operator-level resource functions. The processing element-level resource function represents an application resource model usable for predicting one or more characteristics of the application executed in accordance with a second configuration.

    摘要翻译: 公开了在数据流处理系统中组合应用资源模型的技术。 可以使用应用资源模型来理解由数据流处理系统执行时应用程序将消耗哪些资源。 例如,用于构成数据流处理系统的应用资源模型的方法包括以下步骤。 根据第一配置从数据流处理应用的执行中获得一个或多个运营商级度量。 应用由数据流处理系统的一个或多个节点执行,并且应用由一个或多个由一个或多个运算符组成的处理元件组成。 基于获得的一个或多个运营商级度量,生成一个或多个运营商级资源功能。 基于一个或多个生成的运算符级资源函数生成处理元件级资源功能。 处理元件级资源功能表示可用​​于预测根据第二配置执行的应用的一个或多个特性的应用资源模型。