In an image encoder and an image decoder of an image transmission system, enabling the deterioration of decoded images in image quality to be avoided, the color frame information DCF in separating a predetermined video signal into a luminance component and a color component (chroma component) is extracted prior to the compressive encoding processing of a video signal SSDI1, added after the compressive encoding processing and delivered to transmission means 18A, and moreover the color frame information is extracted prior to the decoding processing of the compressed video signal SSDDI1 obtained from the transmission means 18A and is added after the decoding processing, thereby enabling data on color frame information DCF to be added to the decoded video signal SSDI1 after the decoding processing as they remain invariant free from deformation (loss) and delivered to the transmission means. Thus, a playback video signal (composite video signal) SOUTA can be obtained using the color frame information in the decoded video signal SSDI1 and therefore the deterioration of playback images can be prevented.