An image processing apparatus for processing an input moving image including a plurality of access units arranged every first period. The image processing apparatus includes a motion vector calculation unit which calculates a motion vector of an object included in the input moving image every second period, a motion vector conversion unit which converts the motion vector by multiplying the calculated motion vector by a predetermined gain, and a gain calculation unit which calculates the predetermined gain in accordance with a brightness in a user environment and supplies the predetermined gain to the motion vector conversion unit.