A method of reconstructing a previously occluded area of a video image frame exposed by the motion of a segment within said image frame. In one embodiment the method involves approximating the color values of the pixels in the newly exposed area from the color values of the neighboring image segments. The process is refined by identifying a set of neighboring segments to the exposed area, called fill segments, that most closely resemble the color values of the pixels within the exposed area. These fill segments are then used to reconstruct the color values of the exposed area. In one embodiment, the identities of the fill segments are transmitted to an exemplary decoder.