An improved system for identifying the loudest speech signal in a G.723.1 based audio teleconferencing link is disclosed. The system selects the loudest of several analog audio signals by directly analyzing the encoded G.723.1 bit streams representing those signals, rather than by decoding the encoded speech signal in the G.723.1 bit streams and then re-encoding the signal as a selected output bit stream. The system uses the excitation gain parameters encoded in G.723.1 frames to approximate frame gains for respective bit streams and then estimates a short term speech energy for each bit stream by averaging the approximate frame gains over time. The system then compares the estimated speech energy levels and outputs to each conference participant the signal with the highest estimated speech energy as the next portion of an output signal.