According to an aspect of an embodiment, operations may comprise capturing, at a vehicle as the vehicle travels, LIDAR scans and camera images. The operations may further comprise selecting, at the vehicle as the vehicle travels, a subset of the LIDAR scans and the camera images that are determined to be useful for calibration. The operations may further comprise computing, at the vehicle as the vehicle travels, LIDAR-to-camera transformations for the subset of the LIDAR scans and the camera images using an optimization algorithm. The operations may further comprise calibrating, at the vehicle as the vehicle travels, one or more sensors of the vehicle based on the LIDAR-to-camera transformations.