公开(公告)号：US10725754B2

公开(公告)日：2020-07-28

申请号：US16216155

申请日：2018-12-11

Applicant: Huazhong University of Science and Technology

Inventor： Xuanhua Shi ,  Hai Jin ,  Zhixiang Ke ,  Wenchao Wu

IPC: G06F9/44 ,  G06F8/41 ,  G06F11/34 ,  G06F11/36

Abstract: The present invention relates to a method of memory estimation and configuration optimization for a distributed data processing system involves performing match between an application data stream and a data feature library, wherein the application data stream has received analysis and processing on conditional branches and/or loop bodies of an application code in a Java archive of the application, estimating a memory limit for at least one stage of the application based on the successful matching result, optimizing configuration parameters of the application accordingly, and acquiring static features and/or dynamic features of the application data based on running of the optimized application and performing persistent recording. Opposite to machine-learning-based memory estimation that does not ensure accuracy and fails to provide fine-grained estimation for individual stages, this method uses application analysis and existing data feature to estimate overall memory occupation more precisely and to estimate memory use of individual job stages for more fine-grained configuration optimization.

公开(公告)号：US20190278573A1

公开(公告)日：2019-09-12

申请号：US16216155

申请日：2018-12-11

Applicant: Huazhong University of Science and Technology

Inventor： Xuanhua SHI ,  Hai Jin ,  Zhixiang Ke ,  Wenchao Wu

IPC: G06F8/41 ,  G06F11/34 ,  G06F11/36

Abstract: The present invention relates to a method of memory estimation and configuration optimization for a distributed data processing system involves performing match between an application data stream and a data feature library, wherein the application data stream has received analysis and processing on conditional branches and/or loop bodies of an application code in a Java archive of the application, estimating a memory limit for at least one stage of the application based on the successful matching result, optimizing configuration parameters of the application accordingly, and acquiring static features and/or dynamic features of the application data based on running of the optimized application and performing persistent recording. Opposite to machine-learning-based memory estimation that does not ensure accuracy and fails to provide fine-grained estimation for individual stages, this method uses application analysis and existing data feature to estimate overall memory occupation more precisely and to estimate memory use of individual job stages for more fine-grained configuration optimization.