The method, system, and apparatus of linear temporal reference scheme having non-reference predictive frames is disclosed. In one embodiment, a method of a temporal reference scheme includes creating a non-reference predictive frame and a reference predictive frame from a reference frame, increasing a quantization factor of the non-reference predictive frame by a threshold value, and decreasing the quantization factor of the reference predictive frame by the threshold value. The method may include toggling between the temporal reference scheme and another temporal reference scheme in which there may be no non-reference predictive frames and only reference predictive frames based on a scenario. The method may also include calculating a target bit rate across a communication channel through a bit rate calculation algorithm that considers a bandwidth capacity across the communication channel.