[Object] To provide a control apparatus, an input apparatus, a control system, a control method, and a handheld apparatus that are capable of preventing a deviation between a display position of a pointer and a relative position of the input apparatus when the input apparatus is moved out of a screen from an end portion of the screen, and with which a user can obtain an intuitional operational feeling.[Solving Means] Not only on a real screen (95) but also on a virtual screen (105) set around the real screen (95), coordinates of a virtual pointer (2′) are generated and stored. Accordingly, by moving an input apparatus (1) only by an amount that the input apparatus has moved outside the real screen (95), in an opposite direction, for example, a user can resume a movement of a pointer (2) that has reached a boundary line of the real screen (95), on the real screen (95). As a result, since coordinate values of the pointer are generated in a pseudo absolute coordinate system without being bound by a small screen, the user can operate the input apparatus (1) intuitionally without stress.