公开(公告)号：US11860874B2

公开(公告)日：2024-01-02

申请号：US18051470

申请日：2022-10-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/2455 ,  G06F11/30 ,  G06F7/53 ,  G06F16/27 ,  G06F11/34

CPC classification number: G06F16/24554 ,  G06F7/5324 ,  G06F11/3006 ,  G06F11/3086 ,  G06F11/3433 ,  G06F16/278 ,  G06F2201/835 ,  G06F2201/86

Abstract: Systems and methods are disclosed for processing and executing queries against one or more dataset. As part of processing the query, the system determines whether the query is susceptible to a significantly imbalanced partition. In the event, the query is susceptible to an imbalanced partition, the system monitors the query and determines whether to perform a multi-partitioning determination to avoid a significantly imbalanced partition.

公开(公告)号：US11615104B2

公开(公告)日：2023-03-28

申请号：US16051215

申请日：2018-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/00 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/21 ,  G06F16/951 ,  G06F40/205

Abstract: Systems and methods are disclosed for executing a query that includes an indication to process data managed by an external data system. The system identifies the external data system that manages the data to be processed and generates a subquery for the external data system. The system determines a data ingest estimate and uses the data ingest estimate to generate instructions for one or more worker nodes to receive and process results of the subquery from the external data system.

公开(公告)号：US11615087B2

公开(公告)日：2023-03-28

申请号：US16945587

申请日：2020-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Ryan Delanoy ,  Yujia Wang

IPC: G06F15/16 ,  G06F16/2453 ,  G06F16/2455 ,  G06F9/50 ,  G06F16/248

Abstract: Systems and methods are described for determining a query execution time in a data intake and query system. The system parses a query to identify different portions of the query that are executed by different components of the data intake and query system. The system determines a query execution time for the different portions of the query based on the corresponding components. Based on the query execution time of the different portions for the query, the system determines a query execution time for the query.

公开(公告)号：US11599541B2

公开(公告)日：2023-03-07

申请号：US16398044

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Asha Andrade

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/2458 ,  G06F16/22 ,  G06F16/901 ,  G06F16/242

Abstract: Systems and methods are described for determining a quantity of records generated by a processing task of a query executed in a data intake and query. The system receives a query and identifies a processing task of the query and a quantity of records to be processed according to the query. The system determines the number of records generated by the processing task based on the number of records to be processed and a record generation estimate. The system can allocate compute resources or determine a query execution time for at least a portion of the query based on the determined quantity of records generated.

公开(公告)号：US11586627B2

公开(公告)日：2023-02-21

申请号：US16397970

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Wayne Patterson ,  Srinivas Bobba

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/27 ,  G06F16/22

Abstract: Systems and methods are described for partitioning and reducing records at ingest of a worker node. The worker node receives chunks of data from one or more indexers of a data intake and query system based on the execution of a query by the data intake and query system. The worker node assigns records to different record groups based on the content of the records. The system also assigns the record to a partition of a group of partitions. Record data of the records in a particular partition is combined. The system processes the partitions based on the query.

公开(公告)号：US11550847B1

公开(公告)日：2023-01-10

申请号：US15967567

申请日：2018-04-30

Applicant: Splunk Inc.

Inventor： Alexandros Batsakis ,  Ashish Mathew ,  Christopher Madden Pride ,  Bharath Kishore Reddy Aleti ,  Sourav Pal ,  Arindam Bhattacharjee ,  James Monschke

IPC: G06F16/903 ,  G06F3/06 ,  G06F9/54 ,  G06F16/23 ,  G06F16/901 ,  G06F16/9032

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system identifies buckets that are to be searched. The data intake and query system performs a hash on bucket identifiers of the identified buckets to identify search nodes to search the buckets.

公开(公告)号：US11416528B2

公开(公告)日：2022-08-16

申请号：US15665279

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Asha Andrade

IPC: G06F16/33 ,  G06F16/2458 ,  G06F16/2453 ,  G06F16/2455

Abstract: Systems and methods for a data index and query system that utilize a query acceleration data store. An example method includes receiving a query identifying a set of data to be processed and a manner of processing the set of data. A query processing scheme for obtaining and processing the set of data is defined. First partial results of the query stored in a data store are identified, with the first partial results corresponding to a first portion of the set of data. One or more partitions are dynamically allocated to obtain a second portion of the set of data from different data sources. The second portion of the set of data is processed to obtain second partial results. The first partial results and second partial results are combined. The query is executed based on the query processing scheme.

公开(公告)号：US11321321B2

公开(公告)日：2022-05-03

申请号：US16397968

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Wayne Patterson

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/22

Abstract: Systems and methods are described for processing records associated with a query that identifies an association between two data fields. The system can obtain a chunk of data that includes multiple records based on a query received by a data intake and query system. At least one record can include multiple sub-records that share a field value for at least one field. The system can generate a record from each sub-record and assign the generated records to one or more groups of partitions. The system can combine record data of generated records assigned to one partition of a group of partitions and then combine record data across the group of partitions. The system can process the results of the combination of records across the group of partitions based on the query.

公开(公告)号：US11281706B2

公开(公告)日：2022-03-22

申请号：US15665159

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Kishore Reddy Ramasayam ,  Alexander Douglas James

IPC: G06F16/335 ,  G06F16/26 ,  G06F16/31 ,  G06F16/2458 ,  G06F16/2453

Abstract: Systems and methods are disclosed for processing and executing queries against one or more dataset sources, where the queries identify a set of data to be processed and a manner of processing the set of data. To query the dataset sources, a query coordinator generates a query processing scheme that includes a dynamic allocation of multiple layers of partitions. The query is then executed based on the query processing scheme.

公开(公告)号：US11176208B2

公开(公告)日：2021-11-16

申请号：US16570545

申请日：2019-09-13

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: Disclosed is a technique that can be performed in a distributed computer network. The technique can include a data index and query system that receives a search query, defines a search scheme for applying the search query on distributed data storage systems including an internal data storage system of the data index and query system and an external data storage system. The internal data storage system stores data as time-indexed events including respective segments of raw machine data. The data index and query system can transfer a portion of the search scheme to a search service, which can return search results obtained by application of the search scheme to the distributed data storage systems including the internal data storage system and the external data storage system. Lastly, the search results or data indicative of the search results can be output on a display device to the user.