Abstract:
Ingest health monitoring includes receiving an event stream of events in a data intake and query system to store on at least one storage system and obtaining an event from the event stream. Ingest health monitoring further includes transmitting the event to a selected ingest module queue for the event, updating an output rate indicator counter for the selected ingest module queue when failure to store the event in the ingest module queue occurs, obtaining the event from the selected ingest module queue, processing the event to generate a file for the event, and transmitting the file to the at least one storage system. Ingest health monitoring further includes updating the write failure indicator counter for a storage system of the at least one storage system when failure to transmit to the storage system occurs and updating the user interface based on the output rate indicator counter and the write failure indicator counter.
Abstract:
Systems and methods are disclosed for monitoring features of a computing device of a distributed computing system using a self-monitoring module. The self-monitoring module can include multiple feature-specific monitoring modules and one or more parent nodes for the feature-specific monitoring modules. A feature-specific monitoring module can identify or detect a fault status change, such as a fault condition or fault resolution, for one or more features. Based on the identified fault conditions or fault resolutions, the feature-specific monitoring module can determine an internal status and communicate an updated status to a parent node.
Abstract:
Embodiments of the present disclosure provide solutions for determining an elected search head captain is unqualified for the position, identifying a more qualified search head, and transferring the captain position to the more qualified search head. A method is provided that includes referencing qualification parameters in an elected search head captain, determining whether the newly elected search head captain is qualified for the position based on the parameters, identifying a more qualified search head to be the search head captain if the newly elected search head captain is determined to be unqualified for the position, and transferring the position of captain to the more qualified search head. The qualification parameters may include, for example, a pre-determined static flag set by an administrator of the search environment, and configuration replication status that corresponds to the most recent configuration state of the search head as recorded by the previous search head captain.
Abstract:
Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.
Abstract:
Systems and methods are disclosed for monitoring features of a computing device of a distributed computing system using a self-monitoring module. The self-monitoring module can include multiple feature-specific monitoring modules and one or more parent nodes for the feature-specific monitoring modules. A feature-specific monitoring module can identify or detect a fault status change, such as a fault condition or fault resolution, for one or more features. Based on the identified fault conditions or fault resolutions, the feature-specific monitoring module can determine an internal status and communicate an updated status to a parent node.
Abstract:
Embodiments of the present disclosure provide solutions for determining an elected search head captain is unqualified for the position, identifying a more qualified search head, and transferring the captain position to the more qualified search head. A method is provided that includes referencing qualification parameters in an elected search head captain, determining whether the newly elected search head captain is qualified for the position based on the parameters, identifying a more qualified search head to be the search head captain if the newly elected search head captain is determined to be unqualified for the position, and transferring the position of captain to the more qualified search head. The qualification parameters may include, for example, a pre-determined static flag set by an administrator of the search environment, and configuration replication status that corresponds to the most recent configuration state of the search head as recorded by the previous search head captain.
Abstract:
Embodiments are directed towards a dynamic change evaluation mechanism, whereby items having a detected possible change are scheduled for re-evaluation for possible changes at a higher frequency than items detected to not have previously changed, while those items detected as not to have changed are dynamically scheduled for re-evaluation based on an evaluation backlog that may be in turn based, in part, on a time from when an item is assigned an expiration time to when the item is evaluated. In one embodiment, a possibly changed item may be assigned a new expiration time independent of the evaluation backlog. In another embodiment, if no change is detected, then the item may be assigned a new expiration time as a function of a previous expiration time and on the evaluation backlog.
Abstract:
A method for file system destinations includes obtaining events for storage on one or more of the storage systems. For each event, the method includes extracting at least one field value from the event, comparing the at least one field value to configurations of the storage systems to identify at least one storage system of the plurality of storage systems having a matching configuration, transmitting the event to an ingest module queue for the at least one storage system, selecting a partition for the event based on the at least one field value to obtain a selected partition, mapping the selected partition to a file using a partition mapping, and appending the event to the file on the at least one storage system.
Abstract:
Systems and methods are disclosed for monitoring features of a computing device of a distributed computing system using a self-monitoring module. The self-monitoring module can include multiple feature-specific monitoring modules and one or more parent nodes for the feature-specific monitoring modules. A feature-specific monitoring module can identify or detect a fault status change, such as a fault condition or fault resolution, for one or more features. Based on the identified fault conditions or fault resolutions, the feature-specific monitoring module can determine an internal status and communicate an updated status to a parent node.
Abstract:
Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.