A system and method for using camera capture settings and related metadata to estimate the parameters for encoding a frame of the captured video data and to modify reference frames to accommodate detected camera setting changes. Global brightness and color changes in video capture may be modeled by performing a sequence of transform operations on the reference frames to further improve the coding efficiency of a video coding system.