-
公开(公告)号:US20220382755A1
公开(公告)日:2022-12-01
申请号:US17652620
申请日:2022-02-25
Applicant: Splunk Inc.
Inventor: Alexandros Batsakis , Scott Calvert , Alexander Douglas James , Bei Li , Ashish Mathew , James Monschke , Sogol Moshtaghi , Christopher Madden Pride , Xiaowei Wang
IPC: G06F16/2453 , G06F16/13
Abstract: Systems and methods are disclosed for mapping search nodes to a search head in a data intake and query system based on a tenant identifier in order to execute a query received by the data intake and query system. The mapping may allow same or similar search nodes to be used to execute queries that are associated with a particular tenant identifier, in order to take advantage of caching and local data stored with those search nodes. In some cases, search nodes can be mapped based on the tenant identifier using a hashing algorithm, such as a consistent hashing algorithm.
-
公开(公告)号:US11455314B2
公开(公告)日:2022-09-27
申请号:US16853974
申请日:2020-04-21
Applicant: Splunk Inc.
Inventor: Ashish Mathew
IPC: G06F16/00 , G06F16/248 , G06F21/60 , G06F21/62 , G06F9/50 , G06F16/2455
Abstract: Embodiments of the present disclosure provide a method for performing search queries in a manner that avoids overloading an indexer cluster or indexers with an unwanted or unauthorized high levels of concurrent searches. The method comprises transmitting a slot request from a search head to a cluster master in response to a query, wherein the cluster master is communicatively coupled with an indexer cluster comprising a plurality of indexers. The method further comprises receiving addresses of active indexers in the indexer cluster and a response to the slot request from the cluster master. Responsive to a grant of a slot by the cluster master, the method comprises using the addresses to transmit the query to the active indexers and receiving results of the query from the active indexers. Subsequently, the method comprises releasing the slot to the cluster master.
-
公开(公告)号:US11308061B2
公开(公告)日:2022-04-19
申请号:US16717956
申请日:2019-12-17
Applicant: Splunk Inc.
Inventor: Ashish Mathew
Abstract: Embodiments of the present disclosure provide a method for performing search queries. The method comprises transmitting a list of active indexers in an indexer cluster from a cluster master for receipt by a first search head, wherein the cluster master is communicatively coupled with an indexer cluster comprising a plurality of indexers and the first search head. The method further comprises receiving a first slot request at the cluster master in response to a query from the first search head, wherein the first search head is operable to transmit the query to the active indexers for execution if granted the slot request. Further, the method comprises evaluating a plurality of policies to determine if the first slot request can be granted and responsive to a positive determination, transmitting an authorization token for a slot to the first search head.
-
公开(公告)号:US11157497B1
公开(公告)日:2021-10-26
申请号:US16513555
申请日:2019-07-16
Applicant: Splunk Inc.
Inventor: Alexandros Batsakis , Scott Calvert , Alexander Douglas James , Bei Li , Ashish Mathew , James Monschke , Sogol Moshtaghi , Christopher Madden Pride , Xiaowei Wang
IPC: G06F16/00 , G06F16/2453
Abstract: Systems and methods are disclosed for dynamically assigning a search head or search nodes in a data intake and query system for a query received by the data intake and query system. Existing search heads and search nodes can periodically report their status to the data intake and query system, which can use that information to help determine the need to provision additional search heads and search nodes. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can use the status information for existing search heads and search nodes to dynamically assign a search head and search nodes for the query. Dynamically assigning the search head and search nodes in this manner may provide many benefits, including improved load balancing and resource utilization.
-
公开(公告)号:US10678803B2
公开(公告)日:2020-06-09
申请号:US15582372
申请日:2017-04-28
Applicant: Splunk Inc.
Inventor: Ashish Mathew
IPC: G06F17/00 , G06F16/248 , G06F21/60 , G06F21/62 , G06F9/50 , G06F16/2455
Abstract: Embodiments of the present disclosure provide a method for performing search queries in a manner that avoids overloading an indexer cluster or indexers with an unwanted or unauthorized high levels of concurrent searches. The method comprises transmitting a slot request from a search head to a cluster master in response to a query, wherein the cluster master is communicatively coupled with an indexer cluster comprising a plurality of indexers. The method further comprises receiving addresses of active indexers in the indexer cluster and a response to the slot request from the cluster master. Responsive to a grant of a slot by the cluster master, the method comprises using the addresses to transmit the query to the active indexers and receiving results of the query from the active indexers. Subsequently, the method comprises releasing the slot to the cluster master.
-
公开(公告)号:US11860940B1
公开(公告)日:2024-01-02
申请号:US17233193
申请日:2021-04-16
Applicant: Splunk Inc.
Inventor: Alexandros Batsakis , Ashish Mathew , Christopher Madden Pride , Bharath Kishore Reddy Aleti , Sourav Pal , Arindam Bhattacharjee , James Monschke
IPC: G06F16/901 , G06F16/903 , G06F16/2458
CPC classification number: G06F16/901 , G06F16/2477 , G06F16/90335
Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system uses a search node catalog to identify search nodes that are available to execute the query and uses a bucket catalog to identify buckets to be searched. The data intake and query system executes the query using the identified bucket and search nodes.
-
公开(公告)号:US11620288B2
公开(公告)日:2023-04-04
申请号:US17652620
申请日:2022-02-25
Applicant: Splunk Inc.
Inventor: Alexandros Batsakis , Scott Calvert , Alexander Douglas James , Bei Li , Ashish Mathew , James Monschke , Sogol Moshtaghi , Christopher Madden Pride , Xiaowei Wang
IPC: G06F16/00 , G06F16/2453 , G06F16/13 , G06F11/34 , G06F16/2455
Abstract: Systems and methods are disclosed for mapping search nodes to a search head in a data intake and query system based on a tenant identifier in order to execute a query received by the data intake and query system. The mapping may allow same or similar search nodes to be used to execute queries that are associated with a particular tenant identifier, in order to take advantage of caching and local data stored with those search nodes. In some cases, search nodes can be mapped based on the tenant identifier using a hashing algorithm, such as a consistent hashing algorithm.
-
公开(公告)号:US11436116B1
公开(公告)日:2022-09-06
申请号:US16778511
申请日:2020-01-31
Applicant: Splunk Inc.
Inventor: Alexandros Batsakis , Mehul Goyal , Ashish Mathew , Douglas Rapp , Igor Stojanovski , Eric Woo
IPC: G06F11/00 , G06F11/30 , G06F16/182 , G06F11/32 , G06F16/17
Abstract: Systems and methods are described for improving data availability and/or resiliency of indexers of a data intake and query system. A data intake and query system can index large amounts of data using one or more indexers. An indexer can store a copy of the data that the indexer is assigned to process in the shared storage system, and a cluster master can track the storage of the data and the indexer assigned to process the data. In the event an indexer fails or is otherwise unable to index data that it has been assigned to index, the cluster master can assign one or more second indexers to process the data. The second indexer can download the data from the shared storage system.
-
公开(公告)号:US10545964B2
公开(公告)日:2020-01-28
申请号:US15419883
申请日:2017-01-30
Applicant: Splunk Inc.
Inventor: Sourav Pal , Ashish Mathew , Xiaowei Wang , Christopher Pride
IPC: G06F16/2455 , G06F16/248 , G06F16/951
Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes receiving a search query by a search head, defining a search process for applying the search query to indexers, delegating a first portion of the search process to indexers and a second portion of the search process to intermediary node(s) communicatively coupled to the search head and the indexers. The first portion can define a search scope for obtaining partial search results of the indexers and the second portion can define operations for combining the partial search results by the intermediary node(s) to produce a combination of the partial search results. The search head then receives the combination of the partial search results, and outputs final search results for the search query, where the final search results are based on the combination of the partial search results.
-
公开(公告)号:US10235431B2
公开(公告)日:2019-03-19
申请号:US15011473
申请日:2016-01-29
Applicant: Splunk Inc.
Inventor: Ashish Mathew , Ledion Bitincka , Igor Stojanovski , Dhruva Kumar Bhagi
IPC: G06F17/30
Abstract: Techniques and mechanisms are disclosed to optimize the size of index files to improve use of storage space available to indexers and other components of a data intake and query system. Index files of a data intake and query system may include, among other data, a keyword portion containing mappings between keywords and location references to event data containing the keywords. Optimizing an amount of storage space used by index files may include removing, modifying and/or recreating various components of index files in response to detecting one or more storage conditions related to the event data indexed by the index files. The optimization of index files generally may attempt to manage a tradeoff between an efficiency with which search requests can be processed using the index files and an amount of storage space occupied by the index files.
-
-
-
-
-
-
-
-
-