公开(公告)号：US20240281667A1

公开(公告)日：2024-08-22

申请号：US18571151

申请日：2021-10-18

Applicant: Intel Corporation

Inventor： Guokai MA ,  Jiong GONG ,  Hongzhen LIU

IPC: G06N3/098

CPC classification number: G06N3/098

Abstract: Provided herein are apparatus and methods for batch rebalance in distributed data parallel DNN training. An apparatus includes interface circuitry; and processor circuitry coupled with the interface circuitry, wherein the processor circuitry is to: obtain sorted samples of a mini batch via the interface circuitry, wherein the sorted samples are in an ascend or descend order based on a volume of each of the samples; and assign the sorted samples to each of a plurality of local batches one by one in an order from a first local batch to a last local batch of the plurality of local batches and then from the last local batch to the first local batch until all of the sorted samples are assigned. Other embodiments may also be disclosed and claimed.

公开(公告)号：US20220350863A1

公开(公告)日：2022-11-03

申请号：US17764114

申请日：2019-12-16

Applicant: Intel Corporation

Inventor： Yong Wu ,  Xiaodong LIN ,  Zhong CAO ,  Feng YUAN ,  Hongzhen LIU

IPC: G06F17/16 ,  G06F17/15 ,  G06F12/0864

Abstract: Systems, apparatuses and methods may provide for technology that determines a ratio of floating point instructions to memory read instructions and controls a dimension size of a matrix kernel based at least in part on the ratio. In one example, the matrix kernel conducts an operation between a first matrix and a second matrix and the technology reuses elements of the first matrix for multiple vector lines of the second matrix.