A diagnosis system trains a set of machine-learned diagnosis models that are configured to receive an image of a patient and generate predictions on whether the patient has one or more health conditions. In one embodiment, the set of machine-learned models are trained to generate predictions for images that contain two or more underlying health conditions of the patient. In one instance, the symptoms for the two or more health conditions are shown as two or more overlapping skin abnormalities on the patient. By using the architectures of the set of diagnosis models described herein, the diagnosis system can generate more accurate predictions for images that contain overlapping symptoms for two or more health conditions compared to existing systems.