Introduced here are computer programs and associated computer-implemented techniques for creating visualizations to explain the outputs produced by models designed for object detection. To accomplish this, a graphics editing platform can obtain a reference output that identifies a region of pixels in a digital image that allegedly contains an object. Then, the graphics editing platform can compute the similarity between the reference output and a series of outputs generated by a model upon being applied to masked versions of the digital image. A visualization component can be produced based on the similarity.