In a digital signal processing system, a method for selecting a transform function to apply to an input signal based on characteristics of the signal, and for self-adjusting criteria which are used in selecting a transform function to apply to a subsequent signal. Characteristics are obtained from the signal. The characteristics are compared to adjustable criteria which are used in selecting a transform function. Differing criteria are maintained for the different selectable transform functions. A record is maintained of transform functions selected and the particular characteristics that caused the selection. Based on the ability of a transform function to minimally define the coded signal, an inverse transform function is selected to decode the signal. The criteria used in selecting a transform function to apply to a subsequent signal are adjusted based on a quality measure of the decoded signal and the record of selected transform functions.