A voice activity detection system (100) filters audio input frames (102), on a frame=by-frame basis through a gammatone filterbank (104) to generate filtered gammatone output signals (106). A signal energy calculator (108) takes the filtered gammatone output signals and generates a plurality of energy envelopes. Weighting factors are constructed (112) are applied to each of the energy envelopes thereby producing normalized weighted signal (116), in which voice regions are emphasized and noise regions are minimized. An entropy measurement (118) is taken to extract information from the normalized weighted signals (116) and generate an entropy signal (120). The entropy signal (120) is averaged and compared to an adaptive entropy threshold (122), indicative of a noise floor. Decision logic (124) is used to identifying speech and noise from the comparison of the averaged entropy signal to the adaptive entropy threshold.