A deterioration estimation system according to an aspect of the present disclosure includes: a memory configured to store instructions; and at least one processor configured to execute the instructions to: detect road surface deterioration from a first road surface image; determine, in the first road surface image, a first region where deterioration is detectable and a second region where deterioration is difficult to detect; calculate a degree of deterioration of the first region based on a detection result from the first road surface image; estimate a degree of deterioration of the second region based on a degree of deterioration in road surface deterioration detected from a second road surface image imaged before the first road surface image; and calculate a degree of deterioration of the road surface based on the calculated degree of deterioration of the first region and the estimated degree of deterioration of the second region.