公开(公告)号：US20240411478A1

公开(公告)日：2024-12-12

申请号：US18734229

申请日：2024-06-05

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Xiao Liu ,  Haiyang Shi ,  Hao Wang

IPC: G06F3/06

Abstract: Embodiments of the disclosure provide a method and an apparatus of data processing, an electronic device and a storage medium. The method includes receiving first access data transmitted by at least one client, the first access data representing an instruction log of a remote direct data read instruction transmitted by the client for target data cached in a non-uniform memory access structure; obtaining a data popularity of the target data based on the first access data, the data popularity representing a frequency of the target data accessed by the remote direct data read instruction; and based on the data popularity of the target data, caching the target data to a target location in a data storage unit implemented based on the non-uniform memory access structure, or migrating the target data out of the data storage unit, wherein the target location has a data read-write speed corresponding to the data popularity.

公开(公告)号：US12229118B2

公开(公告)日：2025-02-18

申请号：US18742351

申请日：2024-06-13

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Haiyang Shi ,  Hao Wang

IPC: G06F16/23 ,  G06F7/14

Abstract: The embodiments of the disclosure provide a dada shuffling method, apparatus and device, a computer-readable storage medium and product. The method comprises: acquiring a data shuffling request; acquiring a shuffling request parameter linked list associated with the at least one data to be shuffled based on the data shuffling request; performing a merging operation on shuffling request parameters in the shuffling request parameter linked list according to the data amount of the data segment corresponding to the shuffling request parameter and memory buffer information to obtain at least one target request parameter; and caching the data to be shuffled corresponding to the at least one target request parameter to a predetermined remote direct memory access network card; and distributing respectively data segments associated with at least one data to be shuffled cached in the remote direct memory access network card to a target server of the data segment.

公开(公告)号：US12299320B2

公开(公告)日：2025-05-13

申请号：US18734229

申请日：2024-06-05

Applicant: Beijing Volcano Engine Technology Co., Ltd.

Inventor： Xiao Liu ,  Haiyang Shi ,  Hao Wang

IPC: G06F3/06

Abstract: A method, apparatus, electronic device and storage medium for data caching based on data popularity is provided. In the method, first access data transmitted by at least one client is received. The first access data represents an instruction log of a remote direct data read instruction transmitted by the client for target data cached in a non-uniform memory access structure. A data popularity of the target data is obtained based on the first access data. The data popularity represents a frequency of the target data accessed by the remote direct data read instruction. Based on the data popularity of the target data, the target data is cached to a target location in a data storage unit implemented based on the non-uniform memory access structure. Alternatively, the target data out of the data storage unit is migrated. The target location has a data read-write speed corresponding to the data popularity.