公开(公告)号：US11442935B2

公开(公告)日：2022-09-13

申请号：US16397930

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Asha Andrade

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/2455 ,  G06F9/50 ,  G06F16/2458

Abstract: Systems and methods are described for determining a record generation estimate related to a particular processing task. The system obtains a sample set of data that includes multiple records. The system applies a processing task, such as a transform or regular expression rule to the sample set of data and determines how many records are generated by the processing task. Based on the number of records generated, the system determines a record generation estimate. The system can use the record generation estimate to allocate compute resources or determine a query execution time for at least a portion of the query based on the record generation estimate.

公开(公告)号：US11310313B2

公开(公告)日：2022-04-19

申请号：US17014244

申请日：2020-09-08

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: G06F15/167 ,  H04L67/1087 ,  H04L67/1004 ,  G06F16/951 ,  H04L43/106 ,  H04L43/16 ,  H04L67/02

Abstract: Multi-threaded processing of search responses returned by search peers is disclosed. An example method may include transmitting, by a computer system, a search request to a plurality of search peers of a data aggregation and analysis system; receiving, by a first processing thread, a plurality of data packets from the plurality of search peers; parsing, by a second processing thread operating asynchronously with respect to the first processing thread, one or more data packets of the plurality of data packets, to produce a partial response to the search request; splitting the partial response into two or more fields; and generating, based on the two or more fields of the partial response, an aggregated response to the search request.

公开(公告)号：US11184467B2

公开(公告)日：2021-11-23

申请号：US16668808

申请日：2019-10-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: H04L29/08

Abstract: Systems and methods for multi-thread processing of messages. An example method comprises: receiving, by a first processing thread, a plurality of network packets from a server; processing the plurality of network packets to produce a message; writing the message to a message queue; retrieving, by a second processing thread, the message from the message queue; producing a memory data structure based on the message; placing the memory data structure into a result queue; and responsive to determining that a total size of messages in the message queue exceeds a certain threshold, causing the first processing thread to suspend receiving network packets.

公开(公告)号：US11163758B2

公开(公告)日：2021-11-02

申请号：US15665248

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Alexander Douglas James ,  Sourav Pal ,  Arindam Bhattacharjee ,  Christopher Pride

IPC: G06F16/242 ,  G06F16/22

Abstract: Systems and methods are disclosed for processing queries against an external data source utilizing dynamically allocated partitions operating on one or more worker nodes. The external data source can include data that has not been processed by the system. To query the external data source, a query coordinator can generate a subquery for the external data source based on determined functionality of the data source. The subquery can identify data in the external data source for processing and a manner for processing the data. In addition, the query coordinator can dynamically allocate partitions operating on worker nodes to retrieve and intake results of the subquery. In some cases, number of partitions allocated can be based on a number of partitions supported by the external data source.

公开(公告)号：US11126632B2

公开(公告)日：2021-09-21

申请号：US16051203

申请日：2018-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/2458 ,  G06F16/27 ,  G06F16/21 ,  G06F16/951 ,  G06F40/205

Abstract: Systems and methods are disclosed for executing a query that includes an indication to process data managed by an external data system. The system identifies the external data system that manages the data to be processed, and obtained search configuration data from the external system. The system uses the search configuration data to generate a subquery for the external data system. The system also generates instructions for one or more worker nodes to receive and process results of the subquery from the external data system.

公开(公告)号：US11086869B1

公开(公告)日：2021-08-10

申请号：US16177256

申请日：2018-10-31

Applicant: Splunk Inc.

Inventor： Bharath Kishore Reddy Aleti ,  Alexandros Batsakis ,  Joseph Gabriel Echeverria ,  Alexander Douglas James ,  Sourav Pal ,  Christopher Madden Pride ,  Sai Krishna Sajja ,  Eric Sammer

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/242 ,  G06F16/9535 ,  G06F40/205 ,  G06F9/54

Abstract: Systems and methods are disclosed for interfacing with one or more components of a data intake and query system. The data intake and query system includes a gateway that interfaces between one or more computer-executable applications and one or more components of the data intake and query system. The data intake and query system can include an intake system configured to ingest data, an indexing system configured to generate and store one or more events based on the data, and a query system configured to execute one or more queries. The intake system can include a streaming data processor and at least one ingestion buffer. The indexing system can include at least one containerized indexing node, and the query system can include at least one containerized search node.

公开(公告)号：US11023463B2

公开(公告)日：2021-06-01

申请号：US16146990

申请日：2018-09-28

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F7/00 ,  G06F16/2453 ,  G06F16/242 ,  G06F16/2452 ,  G06F16/2458

Abstract: Systems and methods are disclosed for receiving, at a data intake and query system, a query that includes a subquery that is to be executed at an external data system that supports a different query language than the data intake and query system. The data intake and query system converts the subquery from the query language supported by the external data system to the query language supported by the data intake and query system. The data intake and query system then processes the query including the translated subquery. The translated subquery is then translated back to the language supported by the external data system including any processing or optimizations performed with respect to the subquery.

公开(公告)号：US20210117425A1

公开(公告)日：2021-04-22

申请号：US16657899

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Balaji Rao ,  Jindrich Dinga ,  Kieran Cairney ,  Manuel Martinez ,  Nitilaksha Halakatti ,  Ningxuan He ,  Arindam Bhattacharjee ,  Sourav Pal ,  Alexandros Batsakis

IPC: G06F16/2453 ,  G06F16/2458 ,  H04L9/08 ,  G06F8/61 ,  H04L29/08 ,  H04L12/24

Abstract: Systems and methods are described for establishing and managing components of a distributed computing framework implemented in a data intake and query system. The distributed computing framework may include a master and a plurality of worker nodes. The master may selectively operate on a search head captain that is chosen from the search heads of the data intake and query system. The search head captain may distribute configuration information for the master and the distributed computing framework to the other search heads, which in turn, may distribute that configuration information to indexers of the data intake and query system. Worker nodes may be selectively activated for operation on the indexers based on the configuration information, and the worker nodes may additionally use the configuration information to contact the master and join the distributed computing framework. This approach may provide numerous benefits, including improved security, flexibility in the selection of worker nodes, and redundancy for failures of physical components of the data intake and query system.

公开(公告)号：US10698900B2

公开(公告)日：2020-06-30

申请号：US15714424

申请日：2017-09-25

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James

IPC: G06F16/00 ,  G06F16/2455 ,  G06F16/13 ,  G06F16/23 ,  G06F16/242 ,  G06F16/903 ,  G06F16/901 ,  H04W12/10 ,  H04L29/06

Abstract: Systems and methods are disclosed for generating a distributed execution model with untrusted commands. The system can receive a query, and process the query to identify the untrusted commands. The system can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the system can generate a data structure and include one or more identifiers associated with the data structure in the distributed execution model. The system can distribute the distributed execution model to one or more nodes in a distributed computing environment for execution.

公开(公告)号：US10592563B2

公开(公告)日：2020-03-17

申请号：US15339853

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The disclosed embodiments include a technique to obtain search results from the application of transformation operations on partial search results obtained from across internal and/or external data sources. Examples of transformation operations include arithmetic operations such as an average, mean, count, or the like. Examples of reporting transformations include join operations, statistics, sort, top head. Hence, the search results of a search query can be derived from partial search result rather than include the actual partial search results. In this case, the ordering of the search results may be nonessential. An example of a search query that requires a transformation operation is a “batch” or “reporting” search query. The related disclosed techniques involve obtaining data stored in the bid data ecosystem, and returning that data or data derived from that data.