Methods for processing a set of successive video frames in two passes to determine the number of bidirectional (B) and unidirectional (P) motion compensated frames to be encoded in a video coding system. During the first pass, motion vectors and motion costs are computed for each frame and a derived cost value is computed based on the motion cost of at least one frame. The derived cost value is used to determine the number (NB) of B-frames to be encoded in the set of successive frames. In the second pass, the set of successive frames are encoded where NB frames are encoded as B-frames and some or all motion vectors computed in the first pass are re-used in the second pass. A scene cut detection method is also provided where an impulse-like increase in a ratio of motion costs is monitored.