-
公开(公告)号:US20180293075A1
公开(公告)日:2018-10-11
申请号:US16006878
申请日:2018-06-13
Applicant: HUAWEI TECHNOLOGIES CO., LTD.
Inventor: Haiyan LIU , Jun XU , Qun YU
IPC: G06F9/38
CPC classification number: G06F9/38 , G06F9/5066 , G06F2209/5011
Abstract: A data processing method and apparatus are provided. A system includes a CPU pool and a storage pool, a mapper node and a reducer node respectively run on different CPUs in the CPU pool, and a remote storage area shared by the mapper node and the reducer node is delimited in the storage pool. In this method, the mapper node executes a map task, and stores a data segment obtained by executing the map task, into the remote storage area; and the reducer node directly obtains a data segment to be processed by the reducer node from the remote storage area, and executes a reduce task on the data segment. The method omits some actions in the prior art, and shortens a time for executing the Map/Reduce task.