Techniques related to automatically segmenting video frames into per pixel fidelity object of interest and background regions are discussed. Such techniques include applying tessellation to a video frame to generate feature frames corresponding to the video frame and applying a segmentation network implementing context aware skip connections to an input volume including the feature frames and a context feature volume corresponding to the video frame to generate a segmentation for the video frame.