The present invention relates to an architecture for stack robust fine granularity scalability (SRFGS), more particularly, SRFGS providing simultaneously temporal scalability and SNR scalability. SRFGS first simplifies the RFGS temporal prediction architecture and then generalizes the prediction concept as the following: the quantization error of the previous layer can be inter-predicted by the reconstructed image in the previous time instance of the same layer. With this concept, the RFGS architecture can be extended to multiple layers that forming a stack to improve the temporal prediction efficiency. SRFGS can be optimized at several operating points to fit the requirements of various applications while the fine granularity and error robustness of RFGS are still remained. The experiment results show that SRFGS can improve the performance of RFGS by 0.4 to 3.0 dB in PSNR.