An encoder is disclosed that is partitioned into discrete hardware modules. The discrete modules include multiple re-entry and exit points that allow enhanced control by software. The software can control the discrete modules during the encoding process and make adjustments according to CPU bandwidth and/or user requirements allowing for enhanced quality control and seamless hardware/software operations. In one embodiment, a media stream is received into an encoder that includes a pipeline of multiple hardware stages for encoding. An intermediate result is provided from at least one of the hardware stages to an encoding control module that processes the intermediate result to determine configuration instructions for a next hardware stage in the pipeline. Thus, the encoding process can be modified dynamically through hardware and software interactions as the media stream progresses through the pipeline of the encoder.