A method is provided for reducing or magnifying the picture size of a video composite signal with the use of fewer memories than a conventional video composite signal processor which processes pixel data after being converted into R, G, B signals. A video composite signal is directly converted into digital data and sampled with a sampling frequency of four times the color sub-carrier and stored in a pair of field buffers to be processed for picture size reduction or magnification. The digital data are reduced or multiplied in units of pairs of data with a phase difference of .pi./2. Color information is reproduced from corresponding two pairs of data of two adjacent fields having a phase difference of .pi..