A hearing device adapted for being worn at or in an ear of a user, comprises a) an input unit comprising at last two input transducers each for converting sound around said hearing device to an electric input signal representing said sound, thereby providing at least two electric input signals; b) a beamformer filter comprising a minimum processing beamformer defined by optimized beamformer weights, the beamformer filter being configured to provide a filtered signal in dependence of said at least two electric input signals and said optimized beamformer weights; c) a reference signal representing sound around said hearing device; d) a performance criterion for said minimum processing beamformer. The minimum processing beamformer is a beamformer that provides the filtered signal with as little modification as possible in terms of a selected distance measure compared to said reference signal, while still fulfilling said performance criterion. The optimized beamformer weights are adaptively determined in dependence of said at least two electric input signals, said reference signal, said distance measure, and said performance criterion. A method of operating a hearing device is further disclosed. The invention may e.g. be used in hearing aids or headsets.