Load balancing, failover, and reliable delivery of data in a data intake and query system

    公开(公告)号:US11809395B1

    公开(公告)日:2023-11-07

    申请号:US17444173

    申请日:2021-07-30

    Applicant: Splunk Inc.

    CPC classification number: G06F16/2228

    Abstract: Systems and methods are described for balancing workloads and reliably delivering data to a plurality of indexing systems in a data intake and query system. A topic-based indexing system load balancer may receive event data from various data sources, each of which may be associated with a topic. The event data may be entirely unparsed, unparsed but divided into events, or parsed into events. The topic-based indexing system load balancer may distribute the received event data on a per-topic or per-event basis to a set of indexing systems, and may distribute topics and events based on the volume received. Unparsed data may be divided into portions, and the topic-based indexing system load balancer may ensure that portions data associated with the same topic are delivered to the same indexer so that events split between two portions may be recombined and indexed.

    Enhancing efficiency of data collection using a discover process

    公开(公告)号:US11934869B1

    公开(公告)日:2024-03-19

    申请号:US17808935

    申请日:2022-06-24

    Applicant: SPLUNK INC.

    Abstract: This technology is directed to facilitating scalable and secure data collection. In particular, scalability of data collection is enabled in a secure manner by, among other things, abstracting a connector(s) to a pod(s) and/or container(s) that executes separate from other data-collecting functionality. For example, an execution manager can initiate deployment of a collect coordinator on a first pod associated with a first job and deployment of a first connector on a second pod associated with a second job separate from the first job of a container-managed platform. The collect coordinator can provide a data collection task to the first connector deployed on the second pod of the second job. The first connector can then obtain the set of data from the data source and provide the set of data to the collect coordinator for providing the set of data to a remote source.

    Efficient message queuing service using multiplexing

    公开(公告)号:US11422873B2

    公开(公告)日:2022-08-23

    申请号:US17158927

    申请日:2021-01-26

    Applicant: SPLUNK Inc.

    Abstract: Embodiments of the present invention are directed to facilitating efficient message queueing. In particular, embodiments herein describe, among other things, a redelivery monitor used to monitor when to redeliver messages, or tasks, for reprocessing based on expiration of a redelivery deadline. In this regard, markers indicating processing states for tasks being processed are read by the redelivery monitor. When the processing state indicates that processing is ongoing, the redelivery deadline is extended such that a message or task is not redelivered for processing while the message or task is being processed.

    EFFICIENT MESSAGE QUEUING SERVICE USING MULTIPLEXING

    公开(公告)号:US20210149751A1

    公开(公告)日:2021-05-20

    申请号:US17158927

    申请日:2021-01-26

    Applicant: SPLUNK Inc.

    Abstract: Embodiments of the present invention are directed to facilitating efficient message queueing. In particular, embodiments herein describe, among other things, a redelivery monitor used to monitor when to redeliver messages, or tasks, for reprocessing based on expiration of a redelivery deadline. In this regard, markers indicating processing states for tasks being processed are read by the redelivery monitor. When the processing state indicates that processing is ongoing, the redelivery deadline is extended such that a message or task is not redelivered for processing while the message or task is being processed.

    Using keep-alive markers to extend redelivery deadlines

    公开(公告)号:US11755390B1

    公开(公告)日:2023-09-12

    申请号:US17873484

    申请日:2022-07-26

    Applicant: Splunk Inc.

    CPC classification number: G06F9/546 G06F9/485 G06F11/3006 G06F11/3476

    Abstract: Embodiments of the present invention are directed to facilitating efficient message queueing. In particular, embodiments herein describe, among other things, a redelivery monitor used to monitor when to redeliver messages, or tasks, for reprocessing based on expiration of a redelivery deadline. In this regard, markers indicating processing states for tasks being processed are read by the redelivery monitor. When the processing state indicates that processing is ongoing, the redelivery deadline is extended such that a message or task is not redelivered for processing while the message or task is being processed.

    Efficient and secure scalable-two-stage data collection

    公开(公告)号:US11416285B1

    公开(公告)日:2022-08-16

    申请号:US16399622

    申请日:2019-04-30

    Applicant: SPLUNK INC.

    Abstract: This technology is directed to facilitating scalable and secure data collection. In particular, scalability of data collection is enabled in a secure manner by, among other things, abstracting a connector(s) to a pod(s) and/or container(s) that executes separate from other data-collecting functionality. For example, an execution manager can initiate deployment of a collect coordinator on a first pod associated with a first job and deployment of a first connector on a second pod associated with a second job separate from the first job of a container-managed platform. The collect coordinator can provide a data collection task to the first connector deployed on the second pod of the second job. The first connector can then obtain the set of data from the data source and provide the set of data to the collect coordinator for providing the set of data to a remote source.

    Efficient message queuing service

    公开(公告)号:US10908977B1

    公开(公告)日:2021-02-02

    申请号:US16592647

    申请日:2019-10-03

    Applicant: SPLUNK INC.

    Abstract: Embodiments of the present invention are directed to facilitating efficient message queueing. In particular, embodiments herein describe, among other things, a redelivery monitor used to monitor when to redeliver messages, or tasks, for reprocessing based on expiration of a redelivery deadline. In this regard, markers indicating processing states for tasks being processed are read by the redelivery monitor. When the processing state indicates that processing is ongoing, the redelivery deadline is extended such that a message or task is not redelivered for processing while the message or task is being processed.

Patent Agency Ranking