公开(公告)号：US20180089278A1

公开(公告)日：2018-03-29

申请号：US15665197

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James ,  Christopher Pride

IPC: G06F17/30 ,  H04L12/26 ,  G06F11/20 ,  H04L29/06 ,  H04L29/08

CPC classification number: G06F16/24568 ,  G06F3/0617 ,  G06F3/065 ,  G06F11/20 ,  G06F11/3409 ,  G06F16/2471 ,  G06F16/27 ,  G06F16/278 ,  G06F16/90335 ,  G06F16/951 ,  H04L43/028 ,  H04L43/08 ,  H04L43/12 ,  H04L43/14 ,  H04L67/1097 ,  H04L69/22

Abstract: Systems and methods are disclosed for processing queries against one or more dataset sources utilizing dynamically allocated partitions operating on one or more worker nodes. The results of the processing are stored in a dataset destination. The queries can identify data in the one or more dataset sources for processing and a manner for processing the data. In addition, the queries can identify the dataset destination for storing results of the query. To process the query, a query coordinator can dynamically allocate partitions operating on worker nodes to retrieve data for processing, process the data, and communicate the data to the dataset sources. In addition, the query coordinator can dynamically allocate partitions based on an identification of the dataset destination.

公开(公告)号：US12013895B2

公开(公告)日：2024-06-18

申请号：US18328607

申请日：2023-06-02

Applicant: Splunk Inc.

Inventor： Alexandros Batsakis ,  Ashish Mathew ,  Christopher Madden Pride ,  Bharath Kishore Reddy Aleti ,  Sourav Pal ,  Arindam Bhattacharjee ,  James Monschke

IPC: G06F16/23 ,  G06F3/06 ,  G06F16/27 ,  G06F16/901 ,  G06F16/903

CPC classification number: G06F16/901 ,  G06F3/0604 ,  G06F3/0644 ,  G06F3/065 ,  G06F3/0652 ,  G06F3/0653 ,  G06F3/0656 ,  G06F3/067 ,  G06F16/23 ,  G06F16/27 ,  G06F16/903

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives raw machine data at an indexing system, and stores at least a portion of the raw machine data in buckets using containerized indexing nodes instantiated in a containerized environment. The data intake and query system stores the buckets in a shared storage system.

公开(公告)号：US11989194B2

公开(公告)日：2024-05-21

申请号：US16657867

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/2458 ,  G06F16/27

CPC classification number: G06F16/2471 ,  G06F16/278

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

公开(公告)号：US11663227B2

公开(公告)日：2023-05-30

申请号：US16051197

申请日：2018-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/00 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/21 ,  G06F16/951 ,  G06F40/205

CPC classification number: G06F16/2471 ,  G06F16/211 ,  G06F16/27 ,  G06F16/951 ,  G06F40/205

Abstract: Systems and methods are disclosed for receiving, at a first data intake and query system, a query that includes an indication to process data managed by another data intake and query system. The first data intake and query system identifies a second data intake and query system that manages the data to be processed and generates a subquery for execution by the second data intake and query system, generates instructions for one or more worker nodes to receive and process results of the subquery from the second data intake and query system, and instructs the worker nodes to provide results of the processing to the first data intake and query system.

公开(公告)号：US11636105B2

公开(公告)日：2023-04-25

申请号：US17196577

申请日：2021-03-09

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/242 ,  G06F16/2458 ,  G06F16/17

Abstract: Systems and methods are disclosed for receiving, at a data intake and query system, a query that includes an indication to process data managed by a third-party data storage and processing system that supports a different query language than the data intake and query system. The data intake and query system identifies a third-party data storage and processing system that manages the data to be processed and generates a subquery for execution by the third-party data storage and processing system, generates instructions for one or more worker nodes to receive and process results of the subquery from the third-party data storage and processing system, and instructs the worker nodes to provide results of the processing to the data intake and query system.

公开(公告)号：US11586692B2

公开(公告)日：2023-02-21

申请号：US17589764

申请日：2022-01-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Alexander Douglas James ,  Sourav Pal

IPC: G06F16/9535 ,  G06F9/54 ,  G06F9/50 ,  G06F16/903 ,  G06F16/2458

Abstract: Systems and methods are disclosed for processing streaming data. The data can come from various sources. Worker nodes can be configured to process the streaming data, without delays that may be caused by indexing the data. The data can be filtered and/or transformed as it is processed. In some cases, data can be stored in a data store without transformation. The data in the data store can be accessed and processed at a later time.

公开(公告)号：US20230049579A1

公开(公告)日：2023-02-16

申请号：US17960286

申请日：2022-10-05

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James

IPC: G06F16/2455 ,  G06F16/13 ,  G06F16/23 ,  G06F16/242 ,  G06F16/903 ,  G06F16/901

Abstract: Systems and methods are disclosed for generating a distributed execution model with untrusted commands. The system can receive a query, and process the query to identify the untrusted commands. The system can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the system can generate a data structure and include one or more identifiers associated with the data structure in the distributed execution model. The system can distribute the distributed execution model to one or more nodes in a distributed computing environment for execution.

公开(公告)号：US11567993B1

公开(公告)日：2023-01-31

申请号：US15967574

申请日：2018-04-30

Applicant: Splunk Inc.

Inventor： Alexandros Batsakis ,  Ashish Mathew ,  Christopher Madden Pride ,  Bharath Kishore Reddy Aleti ,  Sourav Pal ,  Arindam Bhattacharjee ,  James Monschke

IPC: G06F16/901 ,  G06F16/2458 ,  G06F16/903

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system identifies buckets that are to be searched and stores a copy of buckets in memory associated with one or more search nodes. A search node performs a search on buckets residing in its memory.

公开(公告)号：US11494380B2

公开(公告)日：2022-11-08

申请号：US16657899

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Balaji Rao ,  Jindrich Dinga ,  Kieran Cairney ,  Manuel Martinez ,  Nitilaksha Halakatti ,  Ningxuan He ,  Arindam Bhattacharjee ,  Sourav Pal ,  Alexandros Batsakis

IPC: G06F15/16 ,  G06F16/2453 ,  G06F16/2458 ,  H04L9/08 ,  H04L41/0806 ,  H04L67/10 ,  H04L67/52 ,  G06F8/61

Abstract: Systems and methods are described for establishing and managing components of a distributed computing framework implemented in a data intake and query system. The distributed computing framework may include a master and a plurality of worker nodes. The master may selectively operate on a search head captain that is chosen from the search heads of the data intake and query system. The search head captain may distribute configuration information for the master and the distributed computing framework to the other search heads, which in turn, may distribute that configuration information to indexers of the data intake and query system. Worker nodes may be selectively activated for operation on the indexers based on the configuration information, and the worker nodes may additionally use the configuration information to contact the master and join the distributed computing framework. This approach may provide numerous benefits, including improved security, flexibility in the selection of worker nodes, and redundancy for failures of physical components of the data intake and query system.

公开(公告)号：US11481396B2

公开(公告)日：2022-10-25

申请号：US16851979

申请日：2020-04-17

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James

IPC: G06F16/00 ,  G06F16/2455 ,  G06F16/13 ,  G06F16/23 ,  G06F16/242 ,  G06F16/903 ,  G06F16/901 ,  H04W12/10 ,  H04L9/40

Abstract: Systems and methods are disclosed for generating a distributed execution model with untrusted commands. The system can receive a query, and process the query to identify the untrusted commands. The system can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the system can generate a data structure and include one or more identifiers associated with the data structure in the distributed execution model. The system can distribute the distributed execution model to one or more nodes in a distributed computing environment for execution.