Dynamic storage and deferred analysis of data stream events

    公开(公告)号:US11789950B1

    公开(公告)日:2023-10-17

    申请号:US17074445

    申请日:2020-10-19

    Applicant: Splunk Inc.

    CPC classification number: G06F16/24568 G06F16/22 G06F16/254

    Abstract: Systems and methods are described for a streaming data processing system that defers processing of some data based on a determined importance of the data. A streaming data processing system can ingest a data stream that contains multiple events, and can extract data field values from individual events and process the data field values to determine event importance. The streaming data processing system can then do further processing and indexing of high importance events, and can generate a storage prefix for each low importance event that determines where to store the low importance event in a data storage system. The streaming data processing system can then process queries by retrieving the indexed high importance events, and can extract the data field values from a high importance event to determine the storage prefix for retrieving corresponding low importance events from the data storage system.

    Enhancing efficiency of data collection using a discover process

    公开(公告)号:US11934869B1

    公开(公告)日:2024-03-19

    申请号:US17808935

    申请日:2022-06-24

    Applicant: SPLUNK INC.

    Abstract: This technology is directed to facilitating scalable and secure data collection. In particular, scalability of data collection is enabled in a secure manner by, among other things, abstracting a connector(s) to a pod(s) and/or container(s) that executes separate from other data-collecting functionality. For example, an execution manager can initiate deployment of a collect coordinator on a first pod associated with a first job and deployment of a first connector on a second pod associated with a second job separate from the first job of a container-managed platform. The collect coordinator can provide a data collection task to the first connector deployed on the second pod of the second job. The first connector can then obtain the set of data from the data source and provide the set of data to the collect coordinator for providing the set of data to a remote source.

    Workload coordinator for providing state credentials to processing tasks of a data processing pipeline

    公开(公告)号:US11989592B1

    公开(公告)日:2024-05-21

    申请号:US17444170

    申请日:2021-07-30

    Applicant: Splunk Inc.

    CPC classification number: G06F9/505 G06F9/5061 H04L63/0853

    Abstract: Systems and methods are described for implementation by a streaming data processing system of a processing pipeline that obtains data items from one or more data sources. The one or more data sources may include multiple partitions of a topic or multiple topics. The one or more data sources are defined by a data pattern, a group of tenants, or a process. The implementation of the processing pipeline can include identifying the one or more data sources to obtain data items. The processing pipeline can include a plurality of processing tasks to obtain the data items. The streaming data processing system can utilize a workload coordinator to dynamically assign particular data sources to particular processing tasks. The implementation of the processing pipeline can further include executing the plurality of processing tasks to cause processing tasks to obtain data items from assigned data sources and process the data items.

    Efficient and secure scalable-two-stage data collection

    公开(公告)号:US11416285B1

    公开(公告)日:2022-08-16

    申请号:US16399622

    申请日:2019-04-30

    Applicant: SPLUNK INC.

    Abstract: This technology is directed to facilitating scalable and secure data collection. In particular, scalability of data collection is enabled in a secure manner by, among other things, abstracting a connector(s) to a pod(s) and/or container(s) that executes separate from other data-collecting functionality. For example, an execution manager can initiate deployment of a collect coordinator on a first pod associated with a first job and deployment of a first connector on a second pod associated with a second job separate from the first job of a container-managed platform. The collect coordinator can provide a data collection task to the first connector deployed on the second pod of the second job. The first connector can then obtain the set of data from the data source and provide the set of data to the collect coordinator for providing the set of data to a remote source.

Patent Agency Ranking