A hearing device configured to be worn by a user comprises one or more microphones, a processing unit, a speaker, a wireless communication unit, and stress evaluator. The user wearing the hearing device is in an environment defined by an acoustic scene. The microphones receive audio signals from audio sources in the environment and provide them to the processing unit which applies processing parameters to thereby process the audio signals. The speaker provides the processed audio signals to the user. The stress evaluator generates an indication of stress of the user, stress of the user being related to the acoustic scene. The processing unit then decides whether to perform an action on the basis of the received audio signals and the indication of stress of the user.