Provided is a pointing apparatus, a pointer control apparatus, a pointing method, and a pointer control method to a pointing apparatus, a pointer control apparatus, a pointing method, and a pointer control method capable of recognizing image codes included in an image frame using an image sensor to determine a pointing direction, and continuously updating the gain between the displacement of the motion of the pointing apparatus and the displacement of the motion of a displayed pointer. The pointing apparatus includes an image receiving unit sensing image patterns that exist in a sensed region, among all of the image patterns arranged in a display region; an inertial sensor sensing an input motion using at least one of the acceleration and angular velocity that are generated due to the motion; and a coordinate determining unit determining moving coordinates that are moved from the central coordinates of the sensed image pattern by coordinate displacement corresponding to the sensed motion.