In an exemplary assessment system of speech sound listening, an output section presents a speech sound to a user as an auditory stimulation; a positive component determination section determines whether a positive component appears in an event-related potential of an electroencephalogram signal of the user in a range from 600-900 ms from a starting point, the starting point being a point in time at which the speech sound is presented, and outputs an assessment result as to whether the user is listening to the speech sound with strife or not; and a negative component determination section determines whether a negative component appears in the event-related potential in a range from 100-300 ms from a starting point, the starting point being the point in time at which the speech sound is presented, and outputs an assessment result as to whether the user is annoyed by the speech sound or not.