Systems and methods relate to facial video encoding and reconstruction, particularly in ultra-low bandwidth settings. In embodiments, a video conferencing or other streaming application uses automatically tracked feature cropping information. A bounding shape size—used to identify the cropped region—varies and is dynamically determined to maintain a proportion for feature reconstruction, such as resizing in the event of a zoom-in on a face (or other feature of interest) or a zoom-out. The tracking scheme may be used to smooth sudden movements, including lateral ones, to generate more natural transitions between frames. Tracking and cropping information (e.g., size and position of the cropped region) may be embedded within an encoded bitstream as supplemental enhancement information (“SEI”), for eventual decoding by a receiver and for compositing a decoded face at a proper location in the applicable stream.