Systems and methods for beamforming audio signals received from a microphone array. One method includes receiving, with an electronic processor communicatively coupled to the microphone array, a plurality of audio signals from the microphone array. The method includes generating a plurality of beams based on the plurality of audio signals. The method includes detecting that an electronic device is in a body-worn position. The method includes, in response to the device being in the body-worn position, determining at least one restricted direction based on the body-worn position. The method includes generating, for each of the plurality of beams, a likelihood statistic. The method includes, for each of the beams, assigning a weight to the likelihood statistic based on the at least one restricted direction to generate a weighted likelihood statistic. The method includes generating an output audio stream from the plurality of beams based on the weighted likelihood statistic.