A video signal processing apparatus which receives a digital video signal having arrangement of pixels frame by frame as an input, performs compressively coding to the digital video signal, and reproduces compressively coded data, comprises blocking unit for performing block dividing process by dividing the video signal of one frame into first blocks having a first format and second blocks having a second format which is different from the first format, performing block changing process by changing the format of the second blocks to the first format, and producing a converted video signal composed of plural blocks having the first format; and compressive coding unit for performing compressive coding including discrete cosine transform process to the converted video signal produced by the blocking unit.