公开(公告)号：US20220327149A1

公开(公告)日：2022-10-13

申请号：US17655302

申请日：2022-03-17

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Kishore Reddy Ramasayam ,  Alexander Douglas James

IPC: G06F16/335 ,  G06F16/26 ,  G06F16/31 ,  G06F16/2458 ,  G06F16/2453

Abstract: Systems and methods are disclosed for processing and executing queries against one or more dataset sources, where the queries identify a set of data to be processed and a manner of processing the set of data. To query the dataset sources, a query coordinator generates a query processing scheme that includes a dynamic allocation of multiple layers of partitions. The query is then executed based on the query processing scheme.

公开(公告)号：US11151137B2

公开(公告)日：2021-10-19

申请号：US15713976

申请日：2017-09-25

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/2455 ,  G06F7/53 ,  G06F16/27 ,  G06F3/06

Abstract: In an environment where multiple datasets are to be combined, systems and methods are disclosed for allocating a group of data entries from at least one dataset into multiple partitions. For a particular partition, the subgroup in the partition can be combined with data entries from the other dataset. In some cases, groups of data entries from each dataset are assigned to different partitions. For a particular partition, a subgroup is duplicated, some of the data entries of the subgroup are reassigned to other partitions, the subgroup is reformed to include data entries from other partitions, and the reformed subgroup is combined with the subgroup from the other dataset(s).

公开(公告)号：US11023539B2

公开(公告)日：2021-06-01

申请号：US16264430

申请日：2019-01-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: Disclosed is a technique that can be performed in a distributed computer network. The technique can include a data index and query system that receives a search query and defines a search scheme for applying the search query on distributed data storage systems including an internal data storage system of the data intake and query system and an external data storage system communicatively coupled to the data intake and query system over a network. The data index and query system communicates at least a portion of the search scheme to a search service for application on behalf of the data intake and query system, receives from the search service a search result of the search query obtained by application of the search scheme to the distributed data storage systems, and causes the search result or data indicative thereof to be displayed on a display device.

公开(公告)号：US10795884B2

公开(公告)日：2020-10-06

申请号：US15665302

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Ramkumar Chandrasekharan

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/2458 ,  G06F16/33 ,  G06F16/2455

Abstract: Systems and methods are disclosed for processing queries against a common storage utilizing dynamically allocated partitions operating on one or more worker nodes. The common storage can include one or more data stores, which collectively contain a data set divided across multiple buckets of data. To query the common storage, a query coordinator can retrieve metadata regarding the multiple buckets, in order to determine a subset of buckets that are potentially relevant to a query. The query coordinator can then dynamically allocate partitions operating on worker nodes to retrieve and intake individual buckets of the subset into a phased search process. The dynamic allocation can be selected to maximize parallelization of the buckets across partitions, thus increasing a speed at which the common storage can be searched.

公开(公告)号：US10726009B2

公开(公告)日：2020-07-28

申请号：US15665148

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Christopher Pride

IPC: G06F16/2453 ,  G06F16/25 ,  G06F16/2455

Abstract: Systems and methods are disclosed for processing queries against one or more dataset sources. The system tracks query resource data and resource utilization data. The query-resource usage data can indicate resources used to execute queries. The node resource utilization data can indicate current utilization of nodes in the system. Upon receipt of a query that identifies a set of data to be processed and a manner of processing the set of data, the system can use the query-resource usage data and the resource utilization data to define a query processing scheme. The query can then be executed using the query processing scheme. In some cases, the query coordinator can dynamically allocate partitions operating on worker nodes to execute the query.

公开(公告)号：US10698897B2

公开(公告)日：2020-06-30

申请号：US15714133

申请日：2017-09-25

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James

IPC: G06F17/00 ,  G06F16/2455 ,  G06F16/951 ,  G06F16/22 ,  G06F21/62

Abstract: Systems and methods are disclosed for executing a distributed execution model with untrusted commands. The distributed execution model can be distributed to multiple nodes in a distributed computing environment. At least one node can process the distributed execution model to identify an untrusted command. The node can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the node can generate a data structure, and execute at least a portion of the data structure.

公开(公告)号：US10585951B2

公开(公告)日：2020-03-10

申请号：US15339833

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The disclosed embodiments include techniques to obtain ordered search results based on partial search results from across multiple diverse internal and/or external data sources. The ordering of the search results may be with respect to a parameter associated with the partial search results. An example of a parameter includes time. As such, the disclosed technique can provide a time-ordered search result based on partial search results obtained from across multiple internal and/or external data sources. Moreover, the disclosed technique can provide time-ordered search results regardless of whether the partial search results obtained from the diverse data sources are timestamped.

公开(公告)号：US20200050586A1

公开(公告)日：2020-02-13

申请号：US16657872

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Timothy Tully

IPC: G06F16/14 ,  G06F16/13 ,  G06F16/2452 ,  G06F16/17 ,  G06F16/2458

Abstract: Systems and methods are described for executing a query of raw machine data that is stored at a remote data store that may store heterogeneous data. The system can determine the directories or file types that may store event data and may instruct one or more worker nodes to access files that may store events based on the determined directories of file types. Further, the system may exclude files at the remote data store that may not be identified as potentially storing events enabling a query that implicates a heterogeneous data store to be efficiently executed.

公开(公告)号：US20190258632A1

公开(公告)日：2019-08-22

申请号：US16397930

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Asha Andrade

IPC: G06F16/2453 ,  G06F16/2458 ,  G06F9/50 ,  G06F16/2455

Abstract: Systems and methods are described for determining a record generation estimate related to a particular processing task. The system obtains a sample set of data that includes multiple records. The system applies a processing task, such as a transform or regular expression rule to the sample set of data and determines how many records are generated by the processing task. Based on the number of records generated, the system determines a record generation estimate. The system can use the record generation estimate to allocate compute resources or determine a query execution time for at least a portion of the query based on the record generation estimate.

公开(公告)号：US20190258631A1

公开(公告)日：2019-08-22

申请号：US16398031

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Nikhil Roy

IPC: G06F16/2453 ,  G06F16/242

Abstract: Systems and methods are described for scheduling a query for execution. The system receives and parses a query to identify one or more portions of the query. The system determines a resource allocation for each portion of the query, and determines an availability of compute resources for the different portions of the query. Based on the resource allocation and the availability of compute resources, the system schedules the query.