发明申请
- 专利标题: Dynamic Tuning of Memory in MapReduce Systems
- 专利标题(中): MapReduce系统内存的动态调整
-
申请号: US14262056申请日: 2014-04-25
-
公开(公告)号: US20150309731A1公开(公告)日: 2015-10-29
- 发明人: Nicholas C. Fuller , Min Li , Shicong Meng , Jian Tan , Liangzhao Zeng , Li Zhang
- 申请人: International Business Machines Corporation
- 申请人地址: US NY Armonk
- 专利权人: International Business Machines Corporation
- 当前专利权人: International Business Machines Corporation
- 当前专利权人地址: US NY Armonk
- 主分类号: G06F3/06
- IPC分类号: G06F3/06 ; G06F9/48
摘要:
Methods, systems, and computer program products for dynamic tuning of memory in MapReduce systems are provided herein. A method includes analyzing (i) memory usage of a first sub-set of multiple tasks associated with a MapReduce job and (ii) an amount of data utilized across the first sub-set of the multiple tasks; determining a memory size to be allocated to the first sub-set of the multiple tasks based on said analyzing, wherein said memory size minimizes a cost function related to said memory usage and said amount of data utilized; performing a task-wise performance comparison among a second sub-set of the multiple tasks associated with the MapReduce job using the determined memory size to be allocated to the first sub-set of the multiple tasks to generate a set of memory allocation results; and dynamically applying the set of memory allocation results to one or more additional tasks associated with the MapReduce job.
公开/授权文献
- US09582189B2 Dynamic tuning of memory in MapReduce systems 公开/授权日:2017-02-28
信息查询