公开(公告)号：US11023539B2

公开(公告)日：2021-06-01

申请号：US16264430

申请日：2019-01-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: Disclosed is a technique that can be performed in a distributed computer network. The technique can include a data index and query system that receives a search query and defines a search scheme for applying the search query on distributed data storage systems including an internal data storage system of the data intake and query system and an external data storage system communicatively coupled to the data intake and query system over a network. The data index and query system communicates at least a portion of the search scheme to a search service for application on behalf of the data intake and query system, receives from the search service a search result of the search query obtained by application of the search scheme to the distributed data storage systems, and causes the search result or data indicative thereof to be displayed on a display device.

公开(公告)号：US20210058457A1

公开(公告)日：2021-02-25

申请号：US17014244

申请日：2020-09-08

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: H04L29/08 ,  G06F15/167 ,  G06F16/951 ,  H04L12/26

Abstract: Multi-threaded processing of search responses returned by search peers is disclosed. An example method may include transmitting, by a computer system, a search request to a plurality of search peers of a data aggregation and analysis system; receiving, by a first processing thread, a plurality of data packets from the plurality of search peers; parsing, by a second processing thread operating asynchronously with respect to the first processing thread, one or more data packets of the plurality of data packets, to produce a partial response to the search request; splitting the partial response into two or more fields; and generating, based on the two or more fields of the partial response, an aggregated response to the search request.

公开(公告)号：US10795884B2

公开(公告)日：2020-10-06

申请号：US15665302

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Ramkumar Chandrasekharan

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/2458 ,  G06F16/33 ,  G06F16/2455

Abstract: Systems and methods are disclosed for processing queries against a common storage utilizing dynamically allocated partitions operating on one or more worker nodes. The common storage can include one or more data stores, which collectively contain a data set divided across multiple buckets of data. To query the common storage, a query coordinator can retrieve metadata regarding the multiple buckets, in order to determine a subset of buckets that are potentially relevant to a query. The query coordinator can then dynamically allocate partitions operating on worker nodes to retrieve and intake individual buckets of the subset into a phased search process. The dynamic allocation can be selected to maximize parallelization of the buckets across partitions, thus increasing a speed at which the common storage can be searched.

公开(公告)号：US10778761B2

公开(公告)日：2020-09-15

申请号：US16174883

申请日：2018-10-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: G06F15/167 ,  H04L29/08 ,  G06F16/951 ,  H04L12/26

Abstract: Processing of search responses returned by search peers is disclosed. An example method may include transmitting, by a computer system, a search request to a plurality of search peers of a data aggregation and analysis system; receiving a plurality of data packets from the plurality of search peers; parsing one or more data packets of the plurality of data packets, to produce a response to the search request; and splitting the response into two or more fields based on at least one of: a defined set of bit positions or a defined separator.

公开(公告)号：US10726009B2

公开(公告)日：2020-07-28

申请号：US15665148

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Christopher Pride

IPC: G06F16/2453 ,  G06F16/25 ,  G06F16/2455

Abstract: Systems and methods are disclosed for processing queries against one or more dataset sources. The system tracks query resource data and resource utilization data. The query-resource usage data can indicate resources used to execute queries. The node resource utilization data can indicate current utilization of nodes in the system. Upon receipt of a query that identifies a set of data to be processed and a manner of processing the set of data, the system can use the query-resource usage data and the resource utilization data to define a query processing scheme. The query can then be executed using the query processing scheme. In some cases, the query coordinator can dynamically allocate partitions operating on worker nodes to execute the query.

公开(公告)号：US10698897B2

公开(公告)日：2020-06-30

申请号：US15714133

申请日：2017-09-25

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James

IPC: G06F17/00 ,  G06F16/2455 ,  G06F16/951 ,  G06F16/22 ,  G06F21/62

Abstract: Systems and methods are disclosed for executing a distributed execution model with untrusted commands. The distributed execution model can be distributed to multiple nodes in a distributed computing environment. At least one node can process the distributed execution model to identify an untrusted command. The node can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the node can generate a data structure, and execute at least a portion of the data structure.

公开(公告)号：US10585951B2

公开(公告)日：2020-03-10

申请号：US15339833

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The disclosed embodiments include techniques to obtain ordered search results based on partial search results from across multiple diverse internal and/or external data sources. The ordering of the search results may be with respect to a parameter associated with the partial search results. An example of a parameter includes time. As such, the disclosed technique can provide a time-ordered search result based on partial search results obtained from across multiple internal and/or external data sources. Moreover, the disclosed technique can provide time-ordered search results regardless of whether the partial search results obtained from the diverse data sources are timestamped.

公开(公告)号：US20200050586A1

公开(公告)日：2020-02-13

申请号：US16657872

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Timothy Tully

IPC: G06F16/14 ,  G06F16/13 ,  G06F16/2452 ,  G06F16/17 ,  G06F16/2458

Abstract: Systems and methods are described for executing a query of raw machine data that is stored at a remote data store that may store heterogeneous data. The system can determine the directories or file types that may store event data and may instruct one or more worker nodes to access files that may store events based on the determined directories of file types. Further, the system may exclude files at the remote data store that may not be identified as potentially storing events enabling a query that implicates a heterogeneous data store to be efficiently executed.

公开(公告)号：US20190258632A1

公开(公告)日：2019-08-22

申请号：US16397930

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Asha Andrade

IPC: G06F16/2453 ,  G06F16/2458 ,  G06F9/50 ,  G06F16/2455

Abstract: Systems and methods are described for determining a record generation estimate related to a particular processing task. The system obtains a sample set of data that includes multiple records. The system applies a processing task, such as a transform or regular expression rule to the sample set of data and determines how many records are generated by the processing task. Based on the number of records generated, the system determines a record generation estimate. The system can use the record generation estimate to allocate compute resources or determine a query execution time for at least a portion of the query based on the record generation estimate.

公开(公告)号：US20190258631A1

公开(公告)日：2019-08-22

申请号：US16398031

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Nikhil Roy

IPC: G06F16/2453 ,  G06F16/242

Abstract: Systems and methods are described for scheduling a query for execution. The system receives and parses a query to identify one or more portions of the query. The system determines a resource allocation for each portion of the query, and determines an availability of compute resources for the different portions of the query. Based on the resource allocation and the availability of compute resources, the system schedules the query.