Adaptive motion information cost estimation is achieved in processing video information. A transmission cost is estimated that is associated with encoding a motion vector difference (mvd) in motion vectors that describe a motion characteristic of the video information. The mvd is encoded based on minimizing a rate estimation mismatch associated with the motion vectors. The encoding step includes computing a bit count associated with the mvd using CABAC. A value is indexed that corresponds to the cost from one or more entries in the lookup table. The cost relates to context or content characteristics associated with the video information. The lookup table is adaptively updated based on a change in the contextual information or content characteristics. The value is dynamically adjustable based on the change.