A method and apparatus are provided for performing entropy encoding on a fine granular scalability layer. A method of entropy encoding on a plurality of current coefficients of a quality layer among a plurality of quality layers of an image block divided into the plurality of quality layers, includes determining a coding pass with respect to each of the current coefficients, selecting a context model with respect to each of the current coefficients using at least one lower coefficient corresponding to each of the current coefficients if the coding pass is a refinement pass, and performing arithmetic encoding a group of coefficients having a same selected context model among the current coefficients by using the selected context model.