公开(公告)号：US20200026553A1

公开(公告)日：2020-01-23

申请号：US16503145

申请日：2019-07-03

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Weikang GAO ,  Yanlin WANG ,  Yue XING ,  Jianwei ZHANG ,  Yi CHENG

IPC: G06F9/48 ,  G06F9/54 ,  G06F9/4401

Abstract: Embodiments of the present disclosure disclose a method and apparatus for processing data. A specific embodiment of the method comprises: acquiring a to-be-adjusted number of target execution units, the target execution unit referring to a unit executing a target program segment in a stream computing system; adjusting a number of the target execution units in the stream computing system based on the to-be-adjusted number; determining, for a target execution unit in at least one target execution unit after the adjustment, an identifier set corresponding to the target execution unit, an identifier in the identifier set being used to indicate to-be-processed data; and processing, through the target execution unit, the to-be-processed data indicated by the identifier in the corresponding identifier set.

公开(公告)号：US20190213188A1

公开(公告)日：2019-07-11

申请号：US16352576

申请日：2019-03-13

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Jianwei ZHANG ,  Yuncong ZHANG ,  Cong WANG ,  Yao XU ,  Chunyang WEN ,  Xin HUANG ,  Zhan SONG ,  Guanyin ZHU

IPC: G06F16/2453 ,  G06F16/182 ,  G06F16/22 ,  G06F16/242 ,  G06F17/27

CPC classification number: G06F16/24537 ,  G06F9/5066 ,  G06F16/182 ,  G06F16/2228 ,  G06F16/243 ,  G06F17/2705

Abstract: A distributed computing framework and a distributed computing method are provided. A specific embodiment of the distributed computing framework includes: a parsing unit, configured to parse an expression of a distributed computing task, and determine an operator and a field corresponding to the operator; and an operator unit, configured to provide the operator, input parameters of the operator including: the field and a field-type distributed dataset. The type of parameters received and returned by any operator may be the field-type distributed dataset, and any operator may operate on the data corresponding to the field in the field-type distributed dataset. Therefore, any operator needs to be implemented once to realize the reuse of the operator. The distributed computing task is expressed in a simple expression, which simplifies the complexity of writing a distributed computing program with the distributed computing framework used by the user.