A multi-directional remote control system and method is adapted for use with an entertainment system of a type including a display such as a monitor or TV and having display functions employing a mouse type control. The remote controller may be conveniently held in one hand of a user and still provides full mouse type functionality. The remote control system and method images the controller to detect relative motion between the controller and screen. This position information is used for control of a cursor or other GUI interface with automatic control of cursor speed based on detected controller distance from the screen and characteristic hand movement.