The invention discloses a TDI flutter image restoration method based on image block adaptive adjustment. In the TDI flutter remote sensing imaging, the fuzzy kernels of the rows of a flutter image are different because of the line-by-line push-scanning characteristics of a TDI camera, and the line-by-line restoration is used for the flutter image, thereby greatly improving the image restoration effect. When the difference between the fuzzy kernels between rows is smaller, the number of rows of a to-be-restored image block is properly increased at a current line, thereby facilitating the improvement of the restoration effect of the image. When the difference between the fuzzy kernels between rows is bigger, the increase of the number of rows of the image block does not exert remarkable impact on the image restoration effect. However, the increase of the number of rows of the image block will prolong the overall restoration time of the image. Through the construction of a formula of similarity between the fuzzy kernels and the comparison of the difference between the fuzzy kernels between rows of the flutter image, the image block size is used for restoring the current line is determined, and the adaptive adjustment of the image block size is achieved in the line-by-line restoration process of a remote sensing flutter image. The method gives consideration to the restoration time while guaranteeing the image restoration effect.