Systems and methods for training an object detection network are described. Embodiments train an object detection network using a labeled training set, wherein each element of the labeled training set includes an image and ground truth labels for object instances in the image, predict annotation data for a candidate set of unlabeled data using the object detection network, select a sample image from the candidate set using a policy network, generate a labeled sample based on the selected sample image and the annotation data, wherein the labeled sample includes labels for a plurality of object instances in the sample image, and perform additional training on the object detection network based at least in part on the labeled sample.