Systems and methods for generating a composite image from a plurality of source images using a scene dependent multi-band blending operation are provided. The multi-band blending operation implements a filtering operation to reduce blending between objects or surfaces that have natural color and/or brightness differences. More particularly, the typical space invariant upsampling that occurs during multi-band blending can be replaced by a scene dependent filtering operation during upsampling that omits or reduces contributions from pixels associated with different objects in a scene during the multi-band blending process. The scene dependent filtering can be based on scene dependent data, such as height data or slope data, which can be used to identify different objects in a scene.