Apparatus and methods are provided for application layer optimization in a modem data network. The optimization incorporates variable rate transmission across one or more optical data channels. Data throughput is maximized by enabling quality of service profiles on a per transmission channel basis. According to one aspect, a system is provided in which the application layer is aware of and controls the underlying transmission rate and quality of the transmission. This enables the system to fully utilize the transmission capacity of the channel. The application layer may map different applications to different transmission classes of service. The services can be classified based on data throughput rate, guaranteed error rates, latency and cost, among other criteria. This provides flexibility to the application layer to map some loss tolerant applications to a lower cost (per bit) transmission class.