A method, apparatus, and system are disclosed for controlling a set-top box based on natural input from a user through various types of devices. Input devices are used to collect sensor data that represents natural user input for controlling the set-top box. The sensor data is parsed in order to identify the natural user input, and a user interface event is generated based on the natural user input. The user interface event is one that is supported by the set-top box. The user interface event is subsequently communicated to the set-top box.