A conference phone system includes personal headsets and a base unit. The personal headsets individually capture audios of local participants on a conference call (“local audios”) and transmit the local audios in separate and identifiable channels to the base unit. The base unit receives the local audios and transmits the local audios in separate and identifiable audio streams over a network to a network client. For a remote participant on the conference call, the network client reproduces the local audios and indicates one or more participants who are presently speaking. The network client can also virtualize the local audios so that the remote participant can distinguish the participants by their relative positions, whether virtual or actual. The network client uses the audio source identification information of various participants to enable conference features to mute, enhance, or hold private sidebar conversations.