A system for encoding speech includes a speech encoder (106, FIG. 1), a speech recognizer (110), and a difference encoder (108). When the speech recognizer (110) recognizes a word, phoneme or feature within an input speech signal (122), the difference encoder (108) calculates the differences between speech parameters (140, 142) derived by the speech encoder (106) and speech parameters (146, 148) derived by the speech recognizer (110). The difference encoder (108) quantizes the differences (128), which replace corresponding encoder-derived parameters to be transmitted over a channel (130). In one embodiment, the difference encoder representation (128) of the speech parameters consumes fewer bits than the encoder-derived representation (124). Accordingly, the resulting bandwidth consumed by a single channel can be decreased.