公开(公告)号：US10592562B2

公开(公告)日：2020-03-17

申请号：US15339847

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The performance and flexibility of a data intake and query system having capabilities extended by a fabric service (DFS) system can be improved with deployment on a cloud computing platform. The DFS system can extend the capabilities of a data intake and query system by leveraging computing assets from anywhere in a big data ecosystem to collectively execute search queries on diverse data systems regardless of whether data stores are internal of the data intake and query system and/or external data stores that are communicatively coupled to the data intake and query system over a network.

公开(公告)号：US20200065340A1

公开(公告)日：2020-02-27

申请号：US16675026

申请日：2019-11-05

Applicant: Splunk Inc.

Inventor： James Alasdair Robert Hodge ,  Sourav Pal ,  Arindam Bhattacharjee ,  Mustafa Ahamed

IPC: G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The disclosed embodiments also include monitoring and metering services of the data fabric service (DFS) system. Specifically, these services can include techniques for monitoring and metering metrics of the DFS system. The metrics are standards for measuring use or misuse of the DFS system. Examples of the metrics include data or components of the DFS system. For example, a metric can include data stored or communicated by the DFS system or components of the DFS system that are used or reserved for exclusive use by customers. The metrics can be measured with respect to time or computing resources (e.g., CPU utilization, memory usage) of the DFS system. For example, a DFS service can include metering the usage of particular worker nodes by a customer over a threshold period of time.

公开(公告)号：US20200065303A1

公开(公告)日：2020-02-27

申请号：US16657867

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/2458 ,  G06F16/242 ,  G06F16/27 ,  G06F11/07 ,  G06F11/30 ,  G06F9/50

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

公开(公告)号：US20200050607A1

公开(公告)日：2020-02-13

申请号：US16657894

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Wayne Patterson

IPC: G06F16/2453 ,  G06F16/2458 ,  G06F9/48

Abstract: Systems and methods are described for reducing execution time of a query that references external data systems. The system can determine an external data system is capable of processing one or more map or reduce phases of a map-reduce operation. When it is determined that the external data system can process a map or reduce phase, associated operations may be reassigned from the system to the external data system reducing the processing resources used by the system to response to the query and, in some cases, speeding up performance of the query.

公开(公告)号：US10545964B2

公开(公告)日：2020-01-28

申请号：US15419883

申请日：2017-01-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Ashish Mathew ,  Xiaowei Wang ,  Christopher Pride

IPC: G06F16/2455 ,  G06F16/248 ,  G06F16/951

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes receiving a search query by a search head, defining a search process for applying the search query to indexers, delegating a first portion of the search process to indexers and a second portion of the search process to intermediary node(s) communicatively coupled to the search head and the indexers. The first portion can define a search scope for obtaining partial search results of the indexers and the second portion can define operations for combining the partial search results by the intermediary node(s) to produce a combination of the partial search results. The search head then receives the combination of the partial search results, and outputs final search results for the search query, where the final search results are based on the combination of the partial search results.

公开(公告)号：US20200004794A1

公开(公告)日：2020-01-02

申请号：US16570545

申请日：2019-09-13

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: Disclosed is a technique that can be performed in a distributed computer network. The technique can include a data index and query system that receives a search query, defines a search scheme for applying the search query on distributed data storage systems including an internal data storage system of the data index and query system and an external data storage system. The internal data storage system stores data as time-indexed events including respective segments of raw machine data. The data index and query system can transfer a portion of the search scheme to a search service, which can return search results obtained by application of the search scheme to the distributed data storage systems including the internal data storage system and the external data storage system. Lastly, the search results or data indicative of the search results can be output on a display device to the user.

公开(公告)号：US20190147084A1

公开(公告)日：2019-05-16

申请号：US16051304

申请日：2018-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee

IPC: G06F17/30 ,  G06F17/27

Abstract: Systems and methods are disclosed for executing a query that includes an indication to process data managed by an external data system. The system identifies the external data system that manages the data to be processed and generates a subquery for the external data system indicating that the results of the subquery are to be sent to one worker node of multiple worker nodes. The system instructs the one worker node to distribute the results received from the external data system to multiple worker nodes for processing.

公开(公告)号：US20180218045A1

公开(公告)日：2018-08-02

申请号：US15419883

申请日：2017-01-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Ashish Mathew ,  Xiaowei Wang ,  Christopher Pride

IPC: G06F17/30

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes receiving a search query by a search head, defining a search process for applying the search query to indexers, delegating a first portion of the search process to indexers and a second portion of the search process to intermediary node(s) communicatively coupled to the search head and the indexers. The first portion can define a search scope for obtaining partial search results of the indexers and the second portion can define operations for combining the partial search results by the intermediary node(s) to produce a combination of the partial search results. The search head then receives the combination of the partial search results, and outputs final search results for the search query, where the final search results are based on the combination of the partial search results.

公开(公告)号：US09942318B2

公开(公告)日：2018-04-10

申请号：US15334690

申请日：2016-10-26

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: G06F15/167 ,  H04L29/08 ,  G06F17/30 ,  H04L12/26

CPC classification number: H04L67/1087 ,  G06F15/167 ,  G06F17/30864 ,  H04L43/106 ,  H04L43/16 ,  H04L67/02 ,  H04L67/1004

Abstract: Asynchronous processing of messages that are received from multiple servers is disclosed. An example method may include transmitting, by a computer system, a search request to a plurality of search peers of a data aggregation and analysis system. The method may further include receiving a plurality of sub-application layer protocol packets from the plurality of search peers. The method may further include parsing, by a first processing thread of the computer system, one or more sub-application layer protocol packets of the plurality of sub-application layer protocol packets, to produce an application layer message representing a partial response to the search request. The method may further include processing, by a second processing thread of the computer system, the application layer message to produce a memory data structure representing an aggregated response to the search request.

公开(公告)号：US20180089258A1

公开(公告)日：2018-03-29

申请号：US15665187

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F17/30

CPC classification number: G06F16/2425 ,  G06F16/2272 ,  G06F16/24535

Abstract: Systems and methods are disclosed for processing queries against multiple dataset sources. One dataset source can include indexers that index and store data. The system can receive a query that identifies a set of data to be processed and a manner of processing the set of data. The set of data can include a first dataset that is accessible by one or more indexers and a second dataset that is accessible by one or more other dataset sources. A query coordinator can define a query processing scheme for obtaining and processing the set of data that includes a dynamic allocation of multiple layers of partitions. The partitions can operate on multiple worker nodes. The query can then be executed based on the query processing scheme.