A user interface controller of a handheld electronic device (100) that has a camera that generates video images presents (1105) information on a display (105) of the handheld electronic device, processes (1110) the video images to track at least one of a position and orientation of a directing object (260) that is within a field of view (225) of the camera, and modifies (1115) at least one scene presented on the display in response to a track of the directing object. Modification of scenes may include selecting one or more scene objects, moving a cursor object, and adjusting a viewing angle of successive scenes.