Abstract:
In a computer-implemented method for configuring a distributed computer system comprising a plurality of nodes of a plurality of node classes, configuration files for a plurality of nodes of each of the plurality of node classes are stored in a central repository. The configuration files include information representing a desired system state of the distributed computer system, and the distributed computer system operates to keep an actual system state of the distributed computer system consistent with the desired system state. The plurality of node classes includes forwarder nodes for receiving data from an input source, indexer nodes for indexing the data, and search head nodes for searching the data. Responsive to receiving changes to the configuration files, the changes are propagated to nodes of the plurality of nodes impacted by the changes based on a node class of the nodes impacted by the changes.
Abstract:
Techniques are described for managing data within a multi-site clustered data intake and query system. A data intake and query system as described herein generally refers to a system for collecting, retrieving, and analyzing data. In this context, a clustered data intake and query system generally refers to a system environment that is configured to provide data redundancy and other features that improve the availability of data stored by the system. For example, a clustered data intake and query system may be configured to store multiple copies of data stored by the system across multiple components such that recovery from a failure of one or more of the components is possible by using copies of the data stored elsewhere in the cluster.
Abstract:
The disclosed embodiments include a method performed by a data intake and query system. The method includes ingesting collected data including metrics data including key values and numerical values, where each numerical value is indicative of a measured characteristic of a computing resource (e.g., device), and populating a first portion of a metric-series index (msidx) file with the key values and a second portion of the msidx file with the numerical values. The first portion is distinct from the second portion. The method further includes generating metrics from the metrics data, where each metric has dimensions populated with at least some of the key values and has one of the numerical values. The method further includes indexing the metrics by at least one of the dimensions.
Abstract:
The invention is directed towards enabling data volume and data type based licensing of software in a distributed system of a plurality of remote and/or local nodes. The invention enables measuring and optionally restricting the use of software based on one or more provided licenses that restrict the amount and type of data that may be processed by the software. New and older licenses may be added together for a single, bulk entitlement for a given volume of data processing for one or all types of data. Different users in the same enterprise may combine license entitlements too. Also, a new license can be acquired repeatedly, without requiring the issuance of combined licenses by the issuing authority and/or the revocation of prior licenses.
Abstract:
Embodiments are directed towards managing within a cluster environment having a plurality of indexers for data storage using redundancy the data being managed using a generation identifier, such that a primary indexer is designated for a given generation of data. When a master device for the cluster fails, data may continue to be stored using redundancy, and data searches performed may still be performed.
Abstract:
Embodiments are directed towards a dynamic change evaluation mechanism, whereby items having a detected possible change are scheduled for re-evaluation for possible changes at a higher frequency than items detected to not have previously changed, while those items detected as not to have changed are dynamically scheduled for re-evaluation based on an evaluation backlog that may be in turn based, in part, on a time from when an item is assigned an expiration time to when the item is evaluated. In one embodiment, a possibly changed item may be assigned a new expiration time independent of the evaluation backlog. In another embodiment, if no change is detected, then the item may be assigned a new expiration time as a function of a previous expiration time and on the evaluation backlog.
Abstract:
A computing device receives an ingest preview request to preview events to be stored by at least one indexer. Responsive to the ingest preview request, the computing device sends a subscription request to the forwarders. The forwarders receive the subscription request and intercept the events that are being sent to at least one of the indexers. The forwarders then clone matching events to the subscription request and responds to the computing device with the matching events. When the computing device receives the matching events, the computing device adds the matching events to a dispatch directory. The user interface is then populated with events in the dispatch directory.
Abstract:
Multiple storage system event handling includes obtaining multiple events for storage on multiple storage systems. For each of the multiple events, field values from each event are extracted. The field values are matched to configurations of the storage systems to identify a subset of the storage system having a matching configuration. The event is transmitted to the subset. The multiple events are transmitted to heterogeneous subsets.
Abstract:
In accordance with various embodiments of the present disclosure, a first instance of a data intake and query system (DIQS) may receive latency data that indicates latency states of second instances of the DIQS, the latency states indicative of latencies associated with processing of event data by the plurality of second instances. The first instance may then determine overall latency state of the first instance based, at least in part, on determining number or percentage of the first instance and the second instances of the DIQS having one or more particular latency states, and determining whether the number or percentage of the first instance and the f second instances of the DIQS having the one or more particular latency states is equal to or exceeds a threshold. The first instance may then present the overall latency state of the first instance.
Abstract:
A log-to-metrics transformation system includes a log-to-metrics application executing on a processor. The log-to-metrics transformation system receives a format associated with machine data, and further receives, via a first graphical control, a first set of metric identifiers corresponding to a first set of metrics associated with the machine data. The log-to-metrics transformation system generates a first set of mappings between the first set of metric identifiers and a first set of field values included in the machine data. The log-to-metrics transformation system stores the first set of mappings and an association with the format of the machine data. The log-to-metrics transformation system, based on the first set of mappings, causes the first set of field values to be extracted from the machine data. Further, a first metric included in the first set of metrics is determined based on at least a portion of the first set of field values.