Diagnosing slow tasks in distributed computing
Abstract:
Machine learning is utilized to analyze respective execution times of a plurality of tasks in a job performed in a distributed computing system to determine that a subset of the plurality of tasks are straggler tasks in the job, where the distributed computing system includes a plurality of computing devices. A supervised machine-learning algorithm is performed using a set of inputs including performance attributes of the plurality of tasks, where the supervised machine learning algorithm uses labels generated from determination of the set of straggler tasks, the performance attributes include respective attributes of the plurality of tasks observed during performance of the job, and applying the supervised learning algorithm results in identification of a set of rules defining conditions, based on the performance attributes of the plurality of tasks, indicative of which tasks will be straggler tasks in a job. Rule data is generated to describe the set of rules.
Public/Granted literature
Information query
Patent Agency Ranking
0/0