A robot control device includes a feature-point detecting unit that detects, from an image of an object acquired by a visual sensor, the positions of a plurality of feature points on the object in a predetermined cycle; a position/orientation calculating unit that updates, in the predetermined cycle, respective equations of motion of the plurality of feature points on the basis of the detected positions of the plurality of feature points and that calculates the position or orientation of the object on the basis of the detected positions of the plurality of feature points calculated from the updated equations of motion; and a robot-arm-movement control unit that controls the movement of a robot arm so as to follow the object, on the basis of the calculated position or orientation of the object.