公开(公告)号：US20250131005A1

公开(公告)日：2025-04-24

申请号：US18885559

申请日：2024-09-13

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Kejian JU ,  Leilei HU ,  Junling DU

IPC: G06F16/25

Abstract: Embodiments of the present disclosure provide a data processing method, apparatus based on data Lake and data Warehouse integration, and an electronic device. The method includes: pre-storing at least two protocol files; receiving a first metadata acquisition request; determining a target protocol file for processing the first metadata acquisition request from the at least two protocol files according to an engine type of the data processing engine which sends the first metadata acquisition request; and parsing, based on the target protocol file, the first metadata acquisition request and acquiring metadata corresponding to the first metadata acquisition request from the metadata storage space. Therefore, for at least two sets of external protocols, one set of metadata is stored, accordingly, the data storage capacity can be reduced; with the setting of one set of metadata, the metadata accessing rights management can be unified, and the security of data can be improved.

公开(公告)号：US20250124027A1

公开(公告)日：2025-04-17

申请号：US18883999

申请日：2024-09-12

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Leilei HU ,  Kejian JU ,  Yuanpu DING

IPC: G06F16/2453

Abstract: Embodiments of the present application provide a data processing method and apparatus, an electronic device, and a medium. The method includes: receiving a data query request for a first data processing engine; selecting a target data processing engine from a data processing engine candidate set in a front-end node based on the data query request; if the target data processing engine is the first data processing engine, parsing the data query request in the front-end node, and sending a parsing result to a back-end node, so that the back-end node performs data query; if the target data processing engine is a second data processing engine, generating target information in the front-end node based on the data query request, and sending the target information to the second data processing engine, so that the second data processing engine obtains data from the first data processing engine for data query.

公开(公告)号：US20250147801A1

公开(公告)日：2025-05-08

申请号：US18934437

申请日：2024-11-01

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Zhaowei HUANG ,  Kejian JU ,  Leilei HU

IPC: G06F9/48 ,  G06F9/50

Abstract: A method and an apparatus for executing a data processing task in a cluster mixed deployment scenario and a device are provided. A first electronic device is provided with computing nodes of at least two data engines. A specific implementation of the method includes: in response to a first data engine node receiving a first data processing task, determining a first memory occupancy corresponding to the first data processing task, where the first data engine node is deployed on the first electronic device and belongs to a first data engine; and determining whether to execute the first data processing task based on a second memory occupancy corresponding to another data engine node, where the another data engine node is deployed on the first electronic device and belongs to another data engine other than the first data engine in the at least two data engines.

公开(公告)号：US20250131006A1

公开(公告)日：2025-04-24

申请号：US18912475

申请日：2024-10-10

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Kejian JU ,  Leilei HU ,  Yuanpu DING

IPC: G06F16/25 ,  G06F16/245

Abstract: Embodiments of the present disclosure provide a data query method based on on-line analytical processing, an electronic device and a storage medium. The method includes: receiving a first query request, and determining a request type of the first query request; executing a query in a row-based storage portion of an event table of a database in response to the request type being a preset type; and generating a query result based on retrieved target data, wherein the event table includes a column-based storage portion and the row-based storage portion, and the column-based storage portion includes a plurality of data rows; and the row-based storage portion includes a coding column, and each piece of coded information in the coding column includes row data belonging to a same data row in the column-based storage portion.

公开(公告)号：US20240281284A1

公开(公告)日：2024-08-22

申请号：US18571133

申请日：2023-03-09

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Leilei HU ,  Fachuan BAI ,  Xuan LUO

IPC: G06F9/48

CPC classification number: G06F9/4881

Abstract: This application provides a task processing method, a first server may receive a first task processing request sent by a client, and then determine information of a task engine that is used to processes the first task processing request. The information of the task engine may be used to establish a connection to the task engine. The first server may store the information of the task engine and an identifier of the client in correspondence into an external memory. After the first server is faulty, another server may determine the information of the task engine based on the information of the task engine and the identifier of the client that are stored correspondingly in the external memory, and establish a connection to the task engine based on the information of the task engine, to use the task engine to continue to process a task initiated by the client.

公开(公告)号：US20250131014A1

公开(公告)日：2025-04-24

申请号：US18885407

申请日：2024-09-13

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Kejian JU ,  Leilei HU ,  Zhaowei HUANG ,  Yuanpu DING

IPC: G06F16/28 ,  G06F16/22

Abstract: A data processing method based on online analytical processing, an electronic device and a storage medium are provided. The method includes: receiving a data operation request, wherein the data operation request includes a first field; determining a target associated data pair from a set of associated data pairs stored in a row-based manner based on the data operation request, and determining a second field associated with the first field according to the determined target associated data pair; taking a field value corresponding to the second field in a fact table of a first database as a target field value corresponding to the first field, and executing an operation indicated by the data operation request for the target field value; wherein fields belonging to the second data type among the associated data pairs are stored in the fact table.

公开(公告)号：US20250124032A1

公开(公告)日：2025-04-17

申请号：US18827533

申请日：2024-09-06

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Leilei HU ,  Kejian JU ,  Zhaowei HUANG ,  Junling DU

IPC: G06F16/2453 ,  G06F16/22 ,  G06F16/23 ,  G06F16/25

Abstract: A data reading and writing method, apparatus and electronic device are provided. In one embodiment, the method includes: receiving a data reading and writing request for a first database, wherein the first database stores data in a column-oriented storage format, the data reading and writing request is used for reading/writing data from/into a row storage table, the first database includes a front-end node and a back-end node, the front-end node is used for receiving the data reading and writing request and the back-end node is used for storing data; parsing the data reading and writing request in the front-end node and sending a parsing result to the back-end node; and reading/writing data from/into a second database by using the parsing result in the back-end node, wherein the first database calls data from the second database, and the second database stores data in a row-oriented storage format.

公开(公告)号：US20240095246A1

公开(公告)日：2024-03-21

申请号：US18470184

申请日：2023-09-19

Applicant: BEIJING VOLCANO ENGINE TECHNOLOGY CO., LTD

Inventor： Zhaowei HUANG ,  Leilei HU ,  Junling DU ,  Xuan LUO

IPC: G06F16/2455 ,  G06F16/21 ,  G06F16/242

CPC classification number: G06F16/24552 ,  G06F16/219 ,  G06F16/244

Abstract: A data query method based on Doris includes: acquiring history data in a bottom operator in a storage engine architecture of the Doris, and performing an aggregation calculation on the history data to obtain an aggregation result of the history data; caching, by a preset cache system, the aggregation result of the history data; performing an aggregation calculation on real-time data in a newly inserted bottom operator in the Doris, in response to a data query request, to obtain an aggregate result of the real-time data; acquiring the aggregation result of the history data from the preset cache system, and merging the aggregation result of the history data with the aggregation result of the real-time data to obtain merged data; and sending, by a data sending operator in the Doris, the merged data to an upper layer of an execution engine, to output a final data query result.