The present invention is a signal processing device which performs parallel processes A and B efficiently. There is a deviation in the throughputs of the process A and B in processing an audio signal. First to Nth sub signal processing sections have capabilities to complete the process A within a period (N×T). A main signal processing sections has a capability to complete the process B within a period T. Efficient signal processing can be achieved by processing an input digital signal by means of distinct sub signal processing devices one after another and then processing the signal by the main signal processing section.