公开(公告)号：US20180089278A1

公开(公告)日：2018-03-29

申请号：US15665197

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James ,  Christopher Pride

IPC: G06F17/30 ,  H04L12/26 ,  G06F11/20 ,  H04L29/06 ,  H04L29/08

CPC classification number: G06F16/24568 ,  G06F3/0617 ,  G06F3/065 ,  G06F11/20 ,  G06F11/3409 ,  G06F16/2471 ,  G06F16/27 ,  G06F16/278 ,  G06F16/90335 ,  G06F16/951 ,  H04L43/028 ,  H04L43/08 ,  H04L43/12 ,  H04L43/14 ,  H04L67/1097 ,  H04L69/22

Abstract: Systems and methods are disclosed for processing queries against one or more dataset sources utilizing dynamically allocated partitions operating on one or more worker nodes. The results of the processing are stored in a dataset destination. The queries can identify data in the one or more dataset sources for processing and a manner for processing the data. In addition, the queries can identify the dataset destination for storing results of the query. To process the query, a query coordinator can dynamically allocate partitions operating on worker nodes to retrieve data for processing, process the data, and communicate the data to the dataset sources. In addition, the query coordinator can dynamically allocate partitions based on an identification of the dataset destination.

公开(公告)号：US12248484B2

公开(公告)日：2025-03-11

申请号：US16657894

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Wayne Patterson

IPC: G06F16/2458 ,  G06F16/2453

Abstract: Systems and methods are described for reducing execution time of a query that references external data systems. The system can determine an external data system is capable of processing one or more map or reduce phases of a map-reduce operation. When it is determined that the external data system can process a map or reduce phase, associated operations may be reassigned from the system to the external data system reducing the processing resources used by the system to response to the query and, in some cases, speeding up performance of the query.

公开(公告)号：US12204593B2

公开(公告)日：2025-01-21

申请号：US18470251

申请日：2023-09-19

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/21 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/248 ,  G06F16/25 ,  G06F16/27 ,  G06F16/901 ,  G06F16/903 ,  G06F16/9038 ,  G06F16/904 ,  G06F16/951

Abstract: Disclosed is a data fabric service system that can be implemented in a distributed computer network, such as a data intake and query system. The data index and query system can receive a search query and define a search scheme for applying the search query on distributed data storage systems including internal data storage and external data storage. The data index and query system may provide a portion of the search scheme to a search service of the data fabric service system, which can cause worker nodes of the data fabric service system to perform various functions—including applying the search query to the external data storage based on the portion of the search scheme in order to obtain search results.

公开(公告)号：US12141183B2

公开(公告)日：2024-11-12

申请号：US17655302

申请日：2022-03-17

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Kishore Reddy Ramasayam ,  Alexander Douglas James

IPC: G06F16/335 ,  G06F16/2453 ,  G06F16/2458 ,  G06F16/26 ,  G06F16/31

Abstract: Systems and methods are disclosed for processing and executing queries against one or more dataset sources, where the queries identify a set of data to be processed and a manner of processing the set of data. To query the dataset sources, a query coordinator generates a query processing scheme that includes a dynamic allocation of multiple layers of partitions. The query is then executed based on the query processing scheme.

公开(公告)号：US11995079B2

公开(公告)日：2024-05-28

申请号：US18181900

申请日：2023-03-10

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/00 ,  G06F16/17 ,  G06F16/22 ,  G06F16/242 ,  G06F16/2453 ,  G06F16/2458 ,  G06F16/25

CPC classification number: G06F16/24535 ,  G06F16/1734 ,  G06F16/22 ,  G06F16/2425 ,  G06F16/24545 ,  G06F16/2471 ,  G06F16/258

Abstract: Systems and methods are disclosed for receiving, at a data intake and query system, a query that includes an indication to process data managed by a third-party data storage and processing system that supports a different query language than the data intake and query system. The data intake and query system identifies a third-party data storage and processing system that manages the data to be processed and generates a subquery for execution by the third-party data storage and processing system, generates instructions for one or more worker nodes to receive and process results of the subquery from the third-party data storage and processing system, and instructs the worker nodes to provide results of the processing to the data intake and query system.

公开(公告)号：US11797618B2

公开(公告)日：2023-10-24

申请号：US17810165

申请日：2022-06-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

CPC classification number: G06F16/951 ,  G06F16/211 ,  G06F16/212 ,  G06F16/248 ,  G06F16/2455 ,  G06F16/2471 ,  G06F16/252 ,  G06F16/258 ,  G06F16/27 ,  G06F16/904 ,  G06F16/9024 ,  G06F16/9038 ,  G06F16/90335

Abstract: Disclosed is a data fabric service system that can be implemented in a distributed computer network, such as a data intake and query system. The data index and query system can receive a search query and define a search scheme for applying the search query on distributed data storage systems including internal data storage and external data storage. The data index and query system may provide a portion of the search scheme to a search service of the data fabric service system, which can cause worker nodes of the data fabric service system to perform various functions—including applying the search query to the external data storage based on the portion of the search scheme in order to obtain search results.

公开(公告)号：US20230147068A1

公开(公告)日：2023-05-11

申请号：US18051481

申请日：2022-10-31

Applicant: Splunk Inc.

Inventor： Balaji Rao ,  Jindrich Dinga ,  Kieran Cairney ,  Manuel Martinez ,  Nitilaksha Halakatti ,  Ningxuan He ,  Arindam Bhattacharjee ,  Sourav Pal ,  Alexandros Batsakis

IPC: G06F16/2453 ,  G06F16/2458 ,  H04L9/08 ,  H04L41/0806 ,  H04L67/10 ,  G06F8/61 ,  H04L67/52

CPC classification number: G06F16/24547 ,  G06F16/2465 ,  H04L9/0866 ,  H04L41/0806 ,  H04L67/10 ,  G06F8/61 ,  H04L67/52

Abstract: Systems and methods are described for establishing and managing components of a distributed computing framework implemented in a data intake and query system. The distributed computing framework may include a master and a plurality of worker nodes. The master may selectively operate on a search head captain that is chosen from the search heads of the data intake and query system. The search head captain may distribute configuration information for the master and the distributed computing framework to the other search heads, which in turn, may distribute that configuration information to indexers of the data intake and query system. Worker nodes may be selectively activated for operation on the indexers based on the configuration information, and the worker nodes may additionally use the configuration information to contact the master and join the distributed computing framework. This approach may provide numerous benefits, including improved security, flexibility in the selection of worker nodes, and redundancy for failures of physical components of the data intake and query system.

公开(公告)号：US11593377B2

公开(公告)日：2023-02-28

申请号：US16397922

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/00 ,  G06F16/2455 ,  G06F16/2458

Abstract: Systems and methods are described for assigning a processing task from one component of a data intake and query system to a different component of the data intake and query system. As part of processing a query, the system can determine that a particular processing task is to be executed by a particular component of the data intake and query system. Based on the characteristics of the component that is to execute the processing task, the system can assign the task or a supplemental task to one or more other components of the data intake and query system.

公开(公告)号：US20220405331A1

公开(公告)日：2022-12-22

申请号：US17810165

申请日：2022-06-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: Disclosed is a data fabric service system that can be implemented in a distributed computer network, such as a data intake and query system. The data index and query system can receive a search query and define a search scheme for applying the search query on distributed data storage systems including internal data storage and external data storage. The data index and query system may provide a portion of the search scheme to a search service of the data fabric service system, which can cause worker nodes of the data fabric service system to perform various functions—including applying the search query to the external data storage based on the portion of the search scheme in order to obtain search results.

公开(公告)号：US20220327125A1

公开(公告)日：2022-10-13

申请号：US17658792

申请日：2022-04-11

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Nikhil Roy

IPC: G06F16/2453 ,  G06F16/242 ,  G06F16/2458 ,  G06F16/17

Abstract: Systems and methods are described for scheduling a query for execution. The system receives and parses a query to identify one or more portions of the query. The system determines a resource allocation for each portion of the query, and determines an availability of compute resources for the different portions of the query. Based on the resource allocation and the availability of compute resources, the system schedules the query.