A system accesses user data describing characteristics of a user and generates a content item score for each content item of a plurality of content items. The system generates the content item score by applying a machine-learning model to the user data, and then generates a plurality of content bundles. The system also generates a bundle score for each content bundle based on corresponding content item scores for the content item associated with each content bundle, randomly selects a bundle of the plurality of content bundles based on the generated bundle scores, and transmits the randomly selected bundle to a client device associated with the user for display to the user. Finally, the system applies the model to each of the generated training examples and updates the parameters of the model based on the model output.