Systems and methods for a streamlined transcoder architecture. A transcoder system includes an encoder and a decoder. The encoder compares a decoded frame and a encoder reference frame to produce an output stream. The decoder produces the decoded frame including decoder reference frame and the encoder reference frame. The decoded frame is produced from an input stream, and the encoder reference frame is produced from the output stream of the encoder. In one embodiment, the encoder refines motion vectors, quantization, and macroblock type/mode from the input stream for reuse in the output stream. Furthermore, the decoded frames from the input stream can be modified in various ways including changing picture resolution and performing image enhancement on them before encoding.