Various implementations include systems for processing audio signals. In particular implementations, a process includes receiving an audio signal, wherein the audio signal includes a speech component of the user and a noise component; filtering the audio signal with a self-speech filter that utilizes an intrinsic user vector to filter out the speech component, wherein the intrinsic user vector is determined based on a voice input of the user; and outputting a filtered audio signal in which the speech component of the user has been substantially removed from the audio signal.