Provided is a method for controlling a buffer in a transmitter and a receiver of a video transmission system in an environment where a channel transfer rate varies. The method includes: if a channel transfer rate is changed, changing a capacity of a buffer in a transmitter being used in proportion to the c rate, the buffer temporarily storing and outputting the video data to a receiver, or changing the delay time of a buffer in the receiver by gradually changing an operating clock in the receiver.