A method and apparatus for processing an image signal includes an image signal encoding apparatus and an image signal decoding apparatus that use an independent luminance and chrominance image signal to reduce a crosstalk. The independent luminance and chrominance image signal may include a luminance signal and a chrominance signal. The image signal encoding apparatus and the image decoding apparatus may perform down-sampling or up-sampling on the chrominance signal. The image signal encoding apparatus and the image decoding apparatus may increase a compression efficiency of the independent luminance and chrominance image signal, by applying different quantization parameters and different bit-depths to the luminance signal and the chrominance signal, respectively.