The invention provides a decoder and a lossy decoding video image method. Each frame of video image is divided into multiple macro blocks. The method comprises steps that a residual value of the present macro block is added with an in-frame prediction value or an inter-frame prediction value to acquire a d-bit reconstruction value of the present macro block which is cached in a first caching region; in-frame prediction is carried out through reading the in-frame decoded d-bit reconstruction value of an adjacent macro block from the first caching region to acquire an in-frame prediction value of the present macro block; the d-bit reconstruction value of the present macro block is transferred right for n bits to acquire a (d-n)-bit reconstruction value of the present macro block, filtering is carried out, or the d-bit reconstruction value of the present macro block is filtered, the filter d-bit reconstruction value of the present macro block is transferred right for the n bits, so the (d-n)-bit decoded result is acquired and outputted, and the n is a positive integer smaller than the d. According to the method, writing-reading DDR bandwidth and hardware resources can be saved, crashing is prevented, and image quality is guaranteed.