A processor-implemented method with data loading includes: dividing a training data set into a plurality of subsets based on sizes of a plurality of data files included in the training data set; loading, from each of the plurality of subsets, a portion of data files in the subset to a plurality of processors based on a proportion of a number of data files of the plurality of subsets in the subset and a batch size of distributed training; and reallocating, based on sizes of data files loaded to processors in a same group among the plurality of processors, the loaded data files to the processors in the same group.