The disclosure relates to a hearing system comprising a first hearing device and a second hearing device configured to be worn at a respective ear of a user, each hearing device comprising a displacement sensor configured to provide respective displacement data indicative of a rotational displacement and/or a translational displacement of the hearing device, and a processing unit communicatively coupled to the displacement sensors. The disclosure further relates to a corresponding method of operating a hearing system, and a computer-readable medium storing instructions to perform the method.