公开(公告)号：US11934418B2

公开(公告)日：2024-03-19

申请号：US17447620

申请日：2021-09-14

Applicant: Splunk Inc.

Inventor： Ashish Mathew ,  Ledion Bitincka ,  Igor Stojanovski ,  Dhruva Kumar Bhagi

IPC: G06F16/248 ,  G06F16/21 ,  G06F16/22 ,  G06F16/28

CPC classification number: G06F16/248 ,  G06F16/2228 ,  G06F16/285 ,  G06F16/21

Abstract: Techniques and mechanisms are disclosed to optimize the size of index files to improve use of storage space available to indexers and other components of a data intake and query system. Index files of a data intake and query system may include, among other data, a keyword portion containing mappings between keywords and location references to event data containing the keywords. Optimizing an amount of storage space used by index files may include removing, modifying and/or recreating various components of index files in response to detecting one or more storage conditions related to the event data indexed by the index files. The optimization of index files generally may attempt to manage a tradeoff between an efficiency with which search requests can be processed using the index files and an amount of storage space occupied by the index files.

公开(公告)号：US11892996B1

公开(公告)日：2024-02-06

申请号：US16513365

申请日：2019-07-16

Applicant: Splunk Inc.

Inventor： Tameem Anwar ,  Alexandros Batsakis ,  Sai Krishna Sajja ,  Igor Stojanovski ,  Eric Woo

IPC: G06F16/22 ,  G06F16/23 ,  G06F16/245 ,  G06F9/50 ,  G06F11/34

CPC classification number: G06F16/2255 ,  G06F9/50 ,  G06F16/2379 ,  G06F16/245 ,  G06F11/34

Abstract: Systems and methods are described for monitoring indexing nodes, populating and maintaining a resource catalog with relevant information, receiving requests for indexing node availability or assignments, identifying indexing nodes that are available to process data, and/or communicating information relating to available indexing nodes. The system can maintain the resource catalog based on communications with each of the containerized indexing nodes. The system can receive, from a partition manager of a data intake and query system, a request for a containerized indexing node that the partition manager can assign to process data received by the partition manager. The system can identify an available containerized indexing node to process the data. The system can communicate, to the partition manager, an indexing node identifier associated with the available containerized indexing node.

公开(公告)号：US11609913B1

公开(公告)日：2023-03-21

申请号：US17162536

申请日：2021-01-29

Applicant: Splunk Inc.

Inventor： Tameem Anwar ,  Alexandros Batsakis ,  Tianyi Gou ,  Mehul Goyal ,  Ashish Mathew ,  Douglas Rapp ,  Sai Krishna Sajja ,  Anish Shrigondekar ,  Igor Stojanovski ,  Eric Woo ,  Zhenghui Xie ,  Ruochen Zhang ,  Sophia Rui Zhu

IPC: G06F16/00 ,  G06F16/2455 ,  G06F16/248 ,  G06F16/2458

Abstract: A data intake and query system can manage the search of large amounts of data using one or more processing nodes. When a new processing node is added or becomes available, the node coordinator can reassign duties from one or more processing nodes to the new processing node. The node coordinator can initially assign the new processing node one or more groups of data for backup purposes. At a later time, the node coordinator can reassign the new processing node to the one or more groups of data for searching purposes.

公开(公告)号：US11500783B1

公开(公告)日：2022-11-15

申请号：US17382043

申请日：2021-07-21

Applicant: Splunk Inc.

Inventor： Bharath Aleti ,  Alexandros Batsakis ,  Paul J. Lucas ,  Igor Stojanovski

IPC: G06F12/121 ,  G06F16/22 ,  G06F16/2455

Abstract: Systems and methods are disclosed for making space available in a local storage of a data intake and query system. A cache manager of the data intake and query system may determine an amount of storage space of a local data store that is available for use to perform a query. The cache manager may then use one or more eviction policies associated with content stored at the local data store to purge content items to evict from the local storage. The system may then retrieve content for performing the query from a remote storage and store the retrieved content at the local storage.

公开(公告)号：US20220269727A1

公开(公告)日：2022-08-25

申请号：US17646841

申请日：2022-01-03

Applicant: Splunk Inc.

Inventor： Alexandros Batsakis ,  Sourav Pal ,  Sai Krishna Sajja ,  Igor Stojanovski ,  Tameem Anwar ,  Paul J. Lucas ,  Eric Woo ,  Steve Wong

IPC: G06F16/901 ,  G06F3/06 ,  G06F16/23 ,  G06F16/27 ,  G06F16/903

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives raw machine data at an indexing system, and stores at least a portion of the raw machine data in buckets using containerized indexing nodes instantiated in a containerized environment. The data intake and query system stores the buckets in a shared storage system.

公开(公告)号：US11250056B1

公开(公告)日：2022-02-15

申请号：US15967573

申请日：2018-04-30

Applicant: Splunk Inc.

Inventor： Alexandros Batsakis ,  Sourav Pal ,  Sai Krishna Sajja ,  Igor Stojanovski ,  Tameem Anwar ,  Eric Woo ,  Steve Wong

IPC: G06F16/901 ,  G06F3/06 ,  G06F16/23 ,  G06F16/903

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. An indexing system of the data intake and query system receives data from an ingestion buffer that includes a marker that indicates data that is made available to the indexing system. The data intake and query system stores at least a portion of the data in buckets and stores the buckets in a shared storage system. Based on the storage of the buckets in the shared storage system, the indexing system indicates to the ingestion buffer that the marker can be updated.

公开(公告)号：US11138218B2

公开(公告)日：2021-10-05

申请号：US16259975

申请日：2019-01-28

Applicant: Splunk Inc.

Inventor： Ashish Mathew ,  Ledion Bitincka ,  Igor Stojanovski ,  Dhruva Kumar Bhagi

IPC: G06F16/248 ,  G06F16/28 ,  G06F16/22 ,  G06F16/21

Abstract: Techniques and mechanisms are disclosed to optimize the size of index files to improve use of storage space available to indexers and other components of a data intake and query system. Index files of a data intake and query system may include, among other data, a keyword portion containing mappings between keywords and location references to event data containing the keywords. Optimizing an amount of storage space used by index files may include removing, modifying and/or recreating various components of index files in response to detecting one or more storage conditions related to the event data indexed by the index files. The optimization of index files generally may attempt to manage a tradeoff between an efficiency with which search requests can be processed using the index files and an amount of storage space occupied by the index files.

公开(公告)号：US20210042369A1

公开(公告)日：2021-02-11

申请号：US17080032

申请日：2020-10-26

Applicant: SPLUNK Inc.

Inventor： Ledion Bitincka ,  Steve Zhang ,  Igor Stojanovski ,  Stephen Sorkin

IPC: G06F16/951 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/903

Abstract: A search request received at a computer of a search support system is processed by analyzing the received search request to identify request parameters and connecting to a system index of the search support system that is referenced in the request parameters. An external result provider (ERP) process is initiated that establishes communication between the search support system and a data source external to the search support system, for a virtual index referenced in the request parameters. Thus, the ERP process provides an interface between the search support system and external data sources, such as by third parties. The ERP process can operate in a streaming mode (providing realtime search results with minimal processing) and/or a reporting mode (providing results with a greater delay and processing extent) and can switch between modes. The search request results are received from the connected system indexes and the referenced virtual indexes.

公开(公告)号：US20190278868A9

公开(公告)日：2019-09-12

申请号：US15885629

申请日：2018-01-31

Applicant: SPLUNK INC.

Inventor： Ledion Bitincka ,  Steve Zhang ,  Igor Stojanovski ,  Stephen Sorkin

IPC: G06F17/30

Abstract: A search request received at a computer of a search support system is processed by analyzing the received search request to identify request parameters and connecting to a system index of the search support system that is referenced in the request parameters. An external result provider (ERP) process is initiated that establishes communication between the search support system and a data source external to the search support system, for a virtual index referenced in the request parameters. Thus, the ERP process provides an interface between the search support system and external data sources, such as by third parties. The ERP process can operate in a streaming mode (providing real-time search results with minimal processing) and/or a reporting mode (providing results with a greater delay and processing extent) and can switch between modes. The search request results are received from the connected system indexes and the referenced virtual indexes.

公开(公告)号：US20180089290A1

公开(公告)日：2018-03-29

申请号：US15339912

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Thomas Allan Haggie ,  Clint Sharp ,  Alexander Douglas James ,  David Ryan Marquardt ,  Hailun Yan ,  Christopher Pride ,  Vishal Patel ,  Amrittpal Singh Bath ,  Pratiksha Shah ,  Murugan Kandaswamy ,  Steve Yu Zhang ,  Ledion Bitincka ,  David E. Simmen ,  Marc Andre Chene ,  Esguerra Ma Kharisma ,  Igor Stojanovski

IPC: G06F17/30

CPC classification number: G06F16/248 ,  G06F3/0481 ,  G06F16/22 ,  G06F16/2228 ,  G06F16/2255 ,  G06F16/2425 ,  G06F16/2455 ,  G06F16/24568 ,  G06F16/2462 ,  G06F16/2477 ,  G06F16/25 ,  G06F16/285 ,  G06F16/8373 ,  G06F16/901 ,  G06F16/90335 ,  G06F16/9038 ,  G06F16/951 ,  G06F16/9535 ,  G06T11/206 ,  G06T2200/24 ,  H04L43/08 ,  H04L67/02 ,  H04L67/025

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes ingesting each metric including at least one key value and a measured value taken of a computing resource, and storing each metric in an index of a metrics store, where the index defines at least one dimension populated with the at least one key value and a measure populated with the measured value. The method further includes cataloging metadata in a metrics catalog, where the metadata is related to the metrics stored in the metrics store, performing an analysis of metrics data included in the metrics store and/or the metrics catalog to obtain results, and causing display of the results or an indication of the results on a display device.