-
公开(公告)号:US11507585B2
公开(公告)日:2022-11-22
申请号:US17124318
申请日:2020-12-16
发明人: Sandeep Bishnoi , Hoyong Park , Subhrajyoti Choudhury , Yathish Gatty , Devendra Jain , Santosh Kumar , Prabhu Thukkaram
IPC分类号: G06F16/2455 , G06F16/2458
摘要: An event processing system for distributing a continuous query, and processing events in an event stream, is disclosed. The system can synchronize timestamps across all partitions by adding a heartbeat propagator component propagating a heartbeat signal for silent partitions. The system can further include a timestamp tracker component that tracks the maximum observed timestamp across all partitions for each batch, and sends this to each partition. The system can distribute partitions of the events to the worker nodes, and instruct a heartbeat propagator component to generate a heartbeat for a first worker node corresponding to an event in a different partition. The system receives outputs from the worker nodes and combines these into a single result corresponding to a continuous query. The system can further implement a timestamp tracker to select a highest timestamp value of a previous batch, and send the value to the worker nodes.
-
公开(公告)号:US11503107B2
公开(公告)日:2022-11-15
申请号:US16950127
申请日:2020-11-17
发明人: Hoyong Park , Gyorgy Geiszter , Dmitrii Andreevich Strizhikozin , Péter Gordos , Prabhu Thukkaram , Vitaly Bychkov , Dmitrii Markovskii
IPC分类号: H04L67/10 , G06F16/2455 , G06F16/901
摘要: A distributed event processing system is disclosed that receives continuous data streams, registers a continuous query against the data streams, and continuously executes the query as new data appears in the streams. In certain embodiments, the distributed event processing system deploys and executes applications (e.g., event processing applications) by distributing the execution of the application on a cluster of machines within the system. In certain embodiments, the system provides users with the ability to specify logical rules in an application. The system processes the logical rules by generating a DAG of transformations representing the logical rules and converts the DAG of transformations into a logical rules Resilient Distributed Dataset (RDD) DAG of transformations. The system processes events in an event batch against the RDD DAG of transformations to generate a set of output results for the user.
-
公开(公告)号:US11061722B2
公开(公告)日:2021-07-13
申请号:US16653758
申请日:2019-10-15
发明人: Hoyong Park , Sandeep Bishnoi , Prabhu Thukkaram
IPC分类号: G06F9/48 , G06F16/2455
摘要: An event processing system for processing events in an event stream is disclosed. The system is configured for configuring a stream processor to micro-batch incoming events from a stream source. The system is also configured for generating a single timestamp for a micro-batch of the incoming events and/or receiving the micro-batch of the incoming events from the stream source. The system can also be configured for assigning the single timestamp to each event of the micro-batch and/or generating separate timestamp values for each respective event of the micro-batch. In some examples, the system can also be configured for assigning, for each respective event of the micro-batch, an individual one of the separate timestamp values.
-
公开(公告)号:US10902000B2
公开(公告)日:2021-01-26
申请号:US16121490
申请日:2018-09-04
发明人: Sandeep Bishnoi , Hoyong Park , Subhrajyoti Choudhury , Yathish Gatty , Devendra Jain , Santosh Kumar , Prabhu Thukkaram
IPC分类号: G06F16/2455 , G06F16/2458
摘要: An event processing system for distributing a continuous query, and processing events in an event stream, is disclosed. The system can synchronize timestamps across all partitions by adding a heartbeat propagator component propagating a heartbeat signal for silent partitions. The system can further include a timestamp tracker component that tracks the maximum observed timestamp across all partitions for each batch, and sends this to each partition. The system can distribute partitions of the events to the worker nodes, and instruct a heartbeat propagator component to generate a heartbeat for a first worker node corresponding to an event in a different partition. The system receives outputs from the worker nodes and combines these into a single result corresponding to a continuous query. The system can further implement a timestamp tracker to select a highest timestamp value of a previous batch, and send the value to the worker nodes.
-
公开(公告)号:US10831619B2
公开(公告)日:2020-11-10
申请号:US16145575
申请日:2018-09-28
发明人: Hoyong Park , Sandeep Bishnoi , Prabhu Thukkaram
摘要: Techniques for providing fault-tolerant stream processing. An exemplary technique includes writing primary output events to a primary target and secondary output events to one or more secondary targets, where the primary output events are written by a primary server and the secondary output events are written by one or more secondary servers. The technique further includes receiving an election of a new primary server from a synchronization system upon a failure of the primary server, where the new primary server is elected from the one or more secondary servers. The technique further includes determining, by the new primary server, the primary output events that failed to be written to the primary target because of the failure of the primary server, and writing, by the new primary server, the failed primary output events to the primary target using the secondary output events read from the one or more secondary targets.
-
6.
公开(公告)号:US10740196B2
公开(公告)日:2020-08-11
申请号:US16279855
申请日:2019-02-19
IPC分类号: G06F11/00 , G06F11/14 , G06F9/54 , G06F16/11 , G06F16/242
摘要: An event processing system is disclosed that processes events of an event stream and performs the recovery of events during system failure. The recovery of events is performed by generating output sequence numbers for each deterministic output event generated by the event processing system. In an embodiment, the event processing system determines a current output sequence number of a most recently transmitted output event in a output stream of events, a last output sequence number of an output event corresponding to a most recently processed batch of input events and determines a set of one or more output events of the output stream to be transmitted based on the current sequence number and the last output sequence number. The event processing system then transmits the output events related to the application.
-
公开(公告)号:US10713249B2
公开(公告)日:2020-07-14
申请号:US15706329
申请日:2017-09-15
发明人: Hoyong Park , Sandeep Bishnoi , Prabhu Thukkaram , Santosh Kumar , Pavan Advani , Kunal Mulay , Jeffrey Toillion
IPC分类号: G06F17/30 , G06F16/2453 , G06F16/13 , G06F16/23 , G06F16/11 , G06F16/901 , G06F16/9032 , G06F16/2455 , G06F16/903 , G06F11/30
摘要: An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a micro-batch stream of input events, process the input events using the CQL engine to generate a set of output events, generate, using a snapshot management algorithm implemented by the CQL engine, a snapshot of a current state of a system based at least in part on the set of output events related to the application, generate a first directory structure to access snapshot information associated with the snapshot of the current state of the system, generate a second directory structure to generate a list of snapshots associated with the current state of the system, and determine based at least in part on the snapshot management algorithm, a process to get, add, or clean the list of snapshots associated with the current state of the system.
-
公开(公告)号:US20190236070A1
公开(公告)日:2019-08-01
申请号:US16376984
申请日:2019-04-05
IPC分类号: G06F16/2458 , G06F16/2455 , G06F16/2453 , G06F16/2452 , G06F16/9535
CPC分类号: G06F16/2471 , G06F16/2452 , G06F16/24535 , G06F16/24568 , G06F16/9535
摘要: Systems and methods for query decomposition for scalability of continuous query processing are disclosed. In some examples, it is determined whether a query is fully stateful. If it is, the query may be divided into two functions, an aggregate function and a global function. The aggregate functions may be distributed across multiple distributed nodes and executed in parallel. The global function may then receive the results from the distributed nodes and join the results to achieve the fully stateful result.
-
公开(公告)号:US20180075125A1
公开(公告)日:2018-03-15
申请号:US15701019
申请日:2017-09-11
发明人: Herbert Stiel , Hoyong Park , Sandeep Bishnoi , Prabhu Thukkaram
IPC分类号: G06F17/30
CPC分类号: G06F16/24568 , G06F8/35 , G06F9/542 , G06F16/21 , G06F16/2433 , G06F16/24532 , G06F16/248 , G06F16/278 , G06F16/9024 , G06F16/90335
摘要: An event processing system for processing events in an event stream is disclosed. The system is configured for determining a stage for a continuous query language (CQL) query being processed by an event processing system and/or determining a stage type associated with the stage. The system is also configured for determining a transformation to be computed for the stage based at least in part on the stage type and/or determining a classification for the CQL query based at least in part on a plurality of rules. The system can also be configured for generating a transformation in a Directly Acyclic Graph (DAG) of a data transformation pipeline for the stage based at least in part on the partitioning criteria for the stage. In some examples, the system can also be configured for determining a partitioning of the stage based at least in part on the transformation.
-
公开(公告)号:US20180075109A1
公开(公告)日:2018-03-15
申请号:US15705406
申请日:2017-09-15
IPC分类号: G06F17/30
CPC分类号: G06F17/30516 , G06F17/30241 , G06F17/30324 , G06F17/30336 , G06F17/30371 , G06F17/30958 , H04W4/021
摘要: An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a continuous data stream related to an application, convert geometry of the continuous data stream into a relation, track a plurality of moving objects in the continuous data stream, determine a relationship between at least a first object and a second object (e.g., a first moving object and a second moving object in the plurality of moving objects) based at least on the geometry of a continuous data stream and the relation. and perform an action based at least one the determined relationship.
-
-
-
-
-
-
-
-
-