A lighting controller is disclosed that automatically adjusts a lighting device based on user activity. The disclosed lighting controller processes at least one of audio and video information to identify predefined user events that suggest that a lighting device should be activated or adjusted. In one implementation, a number of behavior rules define various user activities or events that suggest that the user would like to activate or adjust a lighting device. Each rule contains one or more conditions, and, optionally, a corresponding action-item that should be performed when the rule is satisfied to activate or adjust the lighting device. Upon detection of a predefined event, the corresponding action, if any, is performed by the lighting controller.