公开(公告)号：US20250131321A1

公开(公告)日：2025-04-24

申请号：US18489503

申请日：2023-10-18

Applicant: Google LLC

Inventor： Wei Yu ,  Sang Xie ,  Hieu Hy Pham ,  Quoc V. Le

IPC: G06N20/00

Abstract: Systems and methods are provided for efficiently calibrating a data mixture for training machine-learned models (e.g., machine-learned sequence processing models, such as transformer-based models). For example, machine-learned models can be trained over a broad dataset that can include multiple different categories of data. The mixture of data categories within the dataset can influence model performance. To improve the performance of machine-learned models, example implementations of the present disclosure can learn a distribution of data categories using a lightweight proxy model before initiating training of a large primary model. In this manner, for instance, example implementations can obtain an improved training data distribution with less computational expense and can leverage the learned training data distribution to better train a large primary model.