公开(公告)号：US11163758B2

公开(公告)日：2021-11-02

申请号：US15665248

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Alexander Douglas James ,  Sourav Pal ,  Arindam Bhattacharjee ,  Christopher Pride

IPC: G06F16/242 ,  G06F16/22

Abstract: Systems and methods are disclosed for processing queries against an external data source utilizing dynamically allocated partitions operating on one or more worker nodes. The external data source can include data that has not been processed by the system. To query the external data source, a query coordinator can generate a subquery for the external data source based on determined functionality of the data source. The subquery can identify data in the external data source for processing and a manner for processing the data. In addition, the query coordinator can dynamically allocate partitions operating on worker nodes to retrieve and intake results of the subquery. In some cases, number of partitions allocated can be based on a number of partitions supported by the external data source.

公开(公告)号：US11126632B2

公开(公告)日：2021-09-21

申请号：US16051203

申请日：2018-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/2458 ,  G06F16/27 ,  G06F16/21 ,  G06F16/951 ,  G06F40/205

Abstract: Systems and methods are disclosed for executing a query that includes an indication to process data managed by an external data system. The system identifies the external data system that manages the data to be processed, and obtained search configuration data from the external system. The system uses the search configuration data to generate a subquery for the external data system. The system also generates instructions for one or more worker nodes to receive and process results of the subquery from the external data system.

公开(公告)号：US11023463B2

公开(公告)日：2021-06-01

申请号：US16146990

申请日：2018-09-28

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F7/00 ,  G06F16/2453 ,  G06F16/242 ,  G06F16/2452 ,  G06F16/2458

Abstract: Systems and methods are disclosed for receiving, at a data intake and query system, a query that includes a subquery that is to be executed at an external data system that supports a different query language than the data intake and query system. The data intake and query system converts the subquery from the query language supported by the external data system to the query language supported by the data intake and query system. The data intake and query system then processes the query including the translated subquery. The translated subquery is then translated back to the language supported by the external data system including any processing or optimizations performed with respect to the subquery.

公开(公告)号：US20210117425A1

公开(公告)日：2021-04-22

申请号：US16657899

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Balaji Rao ,  Jindrich Dinga ,  Kieran Cairney ,  Manuel Martinez ,  Nitilaksha Halakatti ,  Ningxuan He ,  Arindam Bhattacharjee ,  Sourav Pal ,  Alexandros Batsakis

IPC: G06F16/2453 ,  G06F16/2458 ,  H04L9/08 ,  G06F8/61 ,  H04L29/08 ,  H04L12/24

Abstract: Systems and methods are described for establishing and managing components of a distributed computing framework implemented in a data intake and query system. The distributed computing framework may include a master and a plurality of worker nodes. The master may selectively operate on a search head captain that is chosen from the search heads of the data intake and query system. The search head captain may distribute configuration information for the master and the distributed computing framework to the other search heads, which in turn, may distribute that configuration information to indexers of the data intake and query system. Worker nodes may be selectively activated for operation on the indexers based on the configuration information, and the worker nodes may additionally use the configuration information to contact the master and join the distributed computing framework. This approach may provide numerous benefits, including improved security, flexibility in the selection of worker nodes, and redundancy for failures of physical components of the data intake and query system.

公开(公告)号：US10698900B2

公开(公告)日：2020-06-30

申请号：US15714424

申请日：2017-09-25

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James

IPC: G06F16/00 ,  G06F16/2455 ,  G06F16/13 ,  G06F16/23 ,  G06F16/242 ,  G06F16/903 ,  G06F16/901 ,  H04W12/10 ,  H04L29/06

Abstract: Systems and methods are disclosed for generating a distributed execution model with untrusted commands. The system can receive a query, and process the query to identify the untrusted commands. The system can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the system can generate a data structure and include one or more identifiers associated with the data structure in the distributed execution model. The system can distribute the distributed execution model to one or more nodes in a distributed computing environment for execution.

公开(公告)号：US10592563B2

公开(公告)日：2020-03-17

申请号：US15339853

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The disclosed embodiments include a technique to obtain search results from the application of transformation operations on partial search results obtained from across internal and/or external data sources. Examples of transformation operations include arithmetic operations such as an average, mean, count, or the like. Examples of reporting transformations include join operations, statistics, sort, top head. Hence, the search results of a search query can be derived from partial search result rather than include the actual partial search results. In this case, the ordering of the search results may be nonessential. An example of a search query that requires a transformation operation is a “batch” or “reporting” search query. The related disclosed techniques involve obtaining data stored in the bid data ecosystem, and returning that data or data derived from that data.

公开(公告)号：US10592562B2

公开(公告)日：2020-03-17

申请号：US15339847

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The performance and flexibility of a data intake and query system having capabilities extended by a fabric service (DFS) system can be improved with deployment on a cloud computing platform. The DFS system can extend the capabilities of a data intake and query system by leveraging computing assets from anywhere in a big data ecosystem to collectively execute search queries on diverse data systems regardless of whether data stores are internal of the data intake and query system and/or external data stores that are communicatively coupled to the data intake and query system over a network.

公开(公告)号：US20200065340A1

公开(公告)日：2020-02-27

申请号：US16675026

申请日：2019-11-05

Applicant: Splunk Inc.

Inventor： James Alasdair Robert Hodge ,  Sourav Pal ,  Arindam Bhattacharjee ,  Mustafa Ahamed

IPC: G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The disclosed embodiments also include monitoring and metering services of the data fabric service (DFS) system. Specifically, these services can include techniques for monitoring and metering metrics of the DFS system. The metrics are standards for measuring use or misuse of the DFS system. Examples of the metrics include data or components of the DFS system. For example, a metric can include data stored or communicated by the DFS system or components of the DFS system that are used or reserved for exclusive use by customers. The metrics can be measured with respect to time or computing resources (e.g., CPU utilization, memory usage) of the DFS system. For example, a DFS service can include metering the usage of particular worker nodes by a customer over a threshold period of time.

公开(公告)号：US20200065303A1

公开(公告)日：2020-02-27

申请号：US16657867

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/2458 ,  G06F16/242 ,  G06F16/27 ,  G06F11/07 ,  G06F11/30 ,  G06F9/50

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

公开(公告)号：US20200050607A1

公开(公告)日：2020-02-13

申请号：US16657894

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Wayne Patterson

IPC: G06F16/2453 ,  G06F16/2458 ,  G06F9/48

Abstract: Systems and methods are described for reducing execution time of a query that references external data systems. The system can determine an external data system is capable of processing one or more map or reduce phases of a map-reduce operation. When it is determined that the external data system can process a map or reduce phase, associated operations may be reassigned from the system to the external data system reducing the processing resources used by the system to response to the query and, in some cases, speeding up performance of the query.