Pipeline set selection based on duty cycle estimation of pipeline threads

    公开(公告)号:US10853082B1

    公开(公告)日:2020-12-01

    申请号:US16399773

    申请日:2019-04-30

    Applicant: Splunk Inc.

    Abstract: A computer implemented system is described for assigning executable jobs to pipeline sets, whereby the jobs may be network based computer jobs. The assigning includes generating a weight for each pipeline set of multiple pipeline sets to obtain multiple weights. Generating a weight includes obtaining duty cycle metrics for pipeline software threads in the pipeline set. The duty cycle metrics include a measure of an amount of time that a corresponding pipeline thread is executing and actively processing data. Generating the weight further includes determining the weight for the pipeline set based at least in part on the duty cycle metrics. The method further includes assigning a job request to a target pipeline set selected from the pipeline sets according to a weighted random algorithm, wherein the weighted random algorithm uses the weights.

    ADAPTIVE MONITORING AND PROCESSING OF NEW DATA FILES AND CHANGES TO EXISTING DATA FILES

    公开(公告)号:US20190026319A1

    公开(公告)日:2019-01-24

    申请号:US16141913

    申请日:2018-09-25

    Applicant: Splunk Inc.

    Abstract: Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.

    Pipeline set selection based on duty cycle estimation

    公开(公告)号:US11513844B1

    公开(公告)日:2022-11-29

    申请号:US17070110

    申请日:2020-10-14

    Applicant: Splunk Inc.

    Abstract: A computer implemented system is described for assigning executable jobs to pipeline sets, whereby the jobs may be network based computer jobs. The assigning includes generating a weight for each pipeline set of multiple pipeline sets to obtain multiple weights. Generating a weight includes obtaining duty cycle metrics for pipeline software threads in the pipeline set. The duty cycle metrics include a measure of an amount of time that a corresponding pipeline thread is executing and actively processing data. Generating the weight further includes determining the weight for the pipeline set based at least in part on the duty cycle metrics. The method further includes assigning a job request to a target pipeline set selected from the pipeline sets according to a weighted random algorithm, wherein the weighted random algorithm uses the weights.

    File Update Tracking
    5.
    发明申请
    File Update Tracking 有权
    文件更新跟踪

    公开(公告)号:US20150154217A1

    公开(公告)日:2015-06-04

    申请号:US14611156

    申请日:2015-01-30

    Applicant: Splunk Inc.

    CPC classification number: G06F17/30144 G06F17/3015 G06F17/30286

    Abstract: Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.

    Abstract translation: 实施例涉及管理和跟踪多个项目的项目识别以确定项目是否是新的或现有的项目,其中已经预先处理了现有项目。 在一些实施例中,可以生成两个或多个项目标识符。 在一个实施例中,生成两个或多个项目标识符可以包括使用小项目尺寸特征,压缩项目或标识符冲突来分析项目。 可以使用两个或更多个项目标识符来确定该项目是新的还是现有的项目。 在一个实施例中,两个或多个项目标识符可以与关于现有项目的记录进行比较,以确定该项目是新项目还是现有项目。 如果项目是现有项目,则可以进一步处理该项目以确定现有项目是否已经实际改变。

    Duty cycle estimation for job assignment

    公开(公告)号:US11775343B1

    公开(公告)日:2023-10-03

    申请号:US17973420

    申请日:2022-10-25

    Applicant: Splunk Inc.

    CPC classification number: G06F9/4881 G06F9/4806 G06F9/4843

    Abstract: A computer implemented system is described for assigning executable jobs to pipeline sets, whereby the jobs may be network based computer jobs. The assigning includes generating a weight for each pipeline set of multiple pipeline sets to obtain multiple weights. Generating a weight includes obtaining duty cycle metrics for pipeline software threads in the pipeline set. The duty cycle metrics include a measure of an amount of time that a corresponding pipeline thread is executing and actively processing data. Generating the weight further includes determining the weight for the pipeline set based at least in part on the duty cycle metrics. The method further includes assigning a job request to a target pipeline set selected from the pipeline sets according to a weighted random algorithm, wherein the weighted random algorithm uses the weights.

    Adaptive monitoring and processing of new data files and changes to existing data files

    公开(公告)号:US10083190B2

    公开(公告)日:2018-09-25

    申请号:US14014059

    申请日:2013-08-29

    Applicant: Splunk Inc.

    CPC classification number: G06F16/21 G06F16/1734

    Abstract: Embodiments are directed towards a dynamic change evaluation mechanism, whereby items having a detected possible change are scheduled for re-evaluation for possible changes at a higher frequency than items detected to not have previously changed, while those items detected as not to have changed are dynamically scheduled for re-evaluation based on an evaluation backlog that may be in turn based, in part, on a time from when an item is assigned an expiration time to when the item is evaluated. In one embodiment, a possibly changed item may be assigned a new expiration time independent of the evaluation backlog. In another embodiment, if no change is detected, then the item may be assigned a new expiration time as a function of a previous expiration time and on the evaluation backlog.

    Dynamic reassignment in a search and indexing system

    公开(公告)号:US11630695B1

    公开(公告)日:2023-04-18

    申请号:US17163160

    申请日:2021-01-29

    Applicant: Splunk Inc.

    Abstract: Dynamic reassignment of search processes into workload pools includes receiving a search query to search at least one data store, assigning the search query to a first workload pool, and executing the search query using a first hardware resource in the first workload pool, the first hardware resource corresponding to a first portion of a hardware device. Dynamic reassignment further includes receiving, while executing the search query, an update command to move the search query to a second workload pool, moving, while executing the search query, the search query to the second workload pool; and continuing execution of the search query using a second hardware resource in the second workload pool. The second hardware resource corresponds to a second portion of the hardware device.

Patent Agency Ranking