The invention discloses a self-adaptive scalable video coding and decoding structure design method which is based on a region of interest and a scalable video coding and decoding technical system architecture of the region of interest (ROI); the content of a video is segmented by a self-adaptive video segmentation and tracking technology, thereby realizing the separation of the region of interest(ROI) from a background; the time-domain redundancy is eliminated by a regional motion compensation time-domain filter technology of the ROI; information of the region of interest (ROI) can be coded and transmitted prior to the background region by a ROI self-adaptive bit plane lifting technology of regional wavelet, thereby obtaining better visual effect under the low code rate; a regional time-frequency domain factor carries out the distinction between the region of interest (ROI) and the background by a regional self-adaptive template technology; and the self-adaptive code rate distribution of the region of interest (ROI) and the background is realized by a regional self-adaptive code rate control technology.