A broadcast processing apparatus, includes: a receiver which receives an image signal; a storage unit; a decoder which decodes the image signal; a channel selection unit which is provided to select a channel; and a controller which determines whether sequence header information exists in the image signal received through the receiver and stores the sequence header information in the storage unit if the sequence header information exists in the image signal, and controls the decoder to decode the image signal by using sequence header information of the selected channel prestored in the storage unit if the sequence header information does not exist, if a selection signal to select the channel is inputted through the channel selection unit.