Provided are a video encoder, a video coding method, a video decoder, and a video decoding method for transmitting a compressed video signal based on a suitable compression method adaptively selected according to the environment. The video coder includes a first encoding portion that removes temporal and spatial redundancy of input video frames, quantizes transform coefficients generated by removing temporal and spatial redundancies from the input video frames, and generates a bitstream, a second encoding portion that removes spatial and temporal redundancy of input video frames, quantizes transform coefficients generated by removing spatial and temporal redundancies from the input video frames, and generates a bitstream, and a mode selector that compares the bitstreams input from the first encoding portion and the second encoding portion with each other, and outputs only the bitstream selected based on the comparison result. Therefore, video frames decoded with various resolution levels can be restored.