A computer-implemented system and method for performing video compression are described. For example, a method according to one embodiment of the invention comprises: encoding a plurality of video frames or portions thereof according to a first encoding format; transmitting the plurality of encoded video frames or portions to a client device; receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has been successfully received and/or decoded; determining latency associated with communicating with the client device; in response to detecting that one or more video frames or portions thereof have not been successfully received and/or decoded: (1) if the latency is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or (2) if the latency is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof; and transmitting the new video frame or portion thereof to the client device.