A scalable video transmission scheme is provided in which client interaction and video content itself are taken into consideration during transmission. Methods and arrangements are provided to prioritize/classify different types of information according to their importance and to packetize or otherwise arrange the prioritized information in a manner such that lower priority information may be dropped during transmission. Thus, when network congestion occurs or there is not enough network bandwidth to transmit all of the prioritized information about an object, some (e.g., lower priority) information may be dropped at the server or at an intermediate network node to reduce the bit rate. Thus, when the server transmits multiple video objects over a channel of limited bandwidth capacity, the bit rate allocated to each object can be adjusted according to several factors, such as, e.g., information importance and client interaction.