公开(公告)号：US20180293075A1

公开(公告)日：2018-10-11

申请号：US16006878

申请日：2018-06-13

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Haiyan LIU ,  Jun XU ,  Qun YU

IPC: G06F9/38

CPC classification number: G06F9/38 ,  G06F9/5066 ,  G06F2209/5011

Abstract: A data processing method and apparatus are provided. A system includes a CPU pool and a storage pool, a mapper node and a reducer node respectively run on different CPUs in the CPU pool, and a remote storage area shared by the mapper node and the reducer node is delimited in the storage pool. In this method, the mapper node executes a map task, and stores a data segment obtained by executing the map task, into the remote storage area; and the reducer node directly obtains a data segment to be processed by the reducer node from the remote storage area, and executes a reduce task on the data segment. The method omits some actions in the prior art, and shortens a time for executing the Map/Reduce task.