Aspects of the technology described herein relate to collection and display of ultrasound images using an explaining model. A first ultrasound image may be determined to be in a first class and a second ultrasound image that is in a second class may be generated based on the first ultrasound image. The second ultrasound image may be generated by an explaining model. A classification model may classify the first and second ultrasound images in the first and second classes, respectively. Generating the second ultrasound image may include changing one or more portions of the first ultrasound image. The explaining model may also generate a transformed version of the first ultrasound image and a mask image, and the second ultrasound image may be a composite image of the first ultrasound image and the transformed version of the first ultrasound image. The mask image may determine how to generate the composite image.