BUCKET DATA DISTRIBUTION FOR EXPORTING DATA TO WORKER NODES

    公开(公告)号:US20190310977A1

    公开(公告)日:2019-10-10

    申请号:US16398038

    申请日:2019-04-29

    Applicant: Splunk Inc.

    Abstract: Systems and methods are described for exporting bucket data from one or more buckets to one or more worker nodes. The system can identify data from different bucket data from buckets stored in a data intake and query system that is to be processed by one or more worker nodes. The system can allocate one or more execution resources, such as a processing pipeline, to process and export the bucket data from the buckets. The system can assign bucket data corresponding to individual buckets to the execution resource based on a bucket distribution policy. The indexer can export the bucket data to the worker nodes for further processing based on the bucket data-execution resource assignment.

    RECORD EXPANSION AND REDUCTION BASED ON A PROCESSING TASK IN A DATA INTAKE AND QUERY SYSTEM

    公开(公告)号:US20190258636A1

    公开(公告)日:2019-08-22

    申请号:US16397968

    申请日:2019-04-29

    Applicant: Splunk Inc.

    Abstract: Systems and methods are described for processing records associated with a query that identifies an association between two data fields. The system can obtain a chunk of data that includes multiple records based on a query received by a data intake and query system. At least one record can include multiple sub-records that share a field value for at least one field. The system can generate a record from each sub-record and assign the generated records to one or more groups of partitions. The system can combine record data of generated records assigned to one partition of a group of partitions and then combine record data across the group of partitions. The system can process the results of the combination of records across the group of partitions based on the query.

    SEARCH SERVICE FOR A DATA FABRIC SYSTEM
    73.
    发明申请

    公开(公告)号:US20190171677A1

    公开(公告)日:2019-06-06

    申请号:US16264441

    申请日:2019-01-31

    Applicant: Splunk Inc.

    Abstract: Disclosed is a technique that can be performed in a distributed network. The technique can include a search service system that receives an indication of at least a portion of a search scheme to cause worker nodes to obtain search results from distributed data storage systems. The search scheme is defined by a data intake and query system. The search service system defines a search process based on the at least a portion of the search scheme and executes the search process to cause the worker nodes to obtain search results from the distributed data storage systems. The search service system receives a combination of search results based on the search results obtained by the worker nodes from the distributed data storage systems, and causes an output based on the combination of search results obtained by the data intake and query system in accordance with the search scheme.

    CONVERTING AND MODIFYING A SUBQUERY FOR AN EXTERNAL DATA SYSTEM

    公开(公告)号:US20190147085A1

    公开(公告)日:2019-05-16

    申请号:US16146990

    申请日:2018-09-28

    Applicant: Splunk Inc.

    Abstract: Systems and methods are disclosed for receiving, at a data intake and query system, a query that includes a subquery that is to be executed at an external data system that supports a different query language than the data intake and query system. The data intake and query system converts the subquery from the query language supported by the external data system to the query language supported by the data intake and query system. The data intake and query system then processes the query including the translated subquery. The translated subquery is then translated back to the language supported by the external data system including any processing or optimizations performed with respect to the subquery.

    Supporting query languages through distributed execution of query engines

    公开(公告)号:US12118009B2

    公开(公告)日:2024-10-15

    申请号:US16657916

    申请日:2019-10-18

    Applicant: Splunk Inc.

    CPC classification number: G06F16/2471 G06F16/24526

    Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

Patent Agency Ranking