A threshold learning control system for learning a controller of a robot. The system includes a threshold learning module, a regime classifier, and an exploratory controller, each receiving sensory inputs from a sensor system of the robot. The regime classifier determines a control regime based on the received sensor inputs and communicates the control regime to the threshold learning module. The exploratory controller also receives control parameters from the threshold learning module. A control arbiter receives commands from the exploratory controller and limits from the threshold learning module, The control arbiter issues modified commands based on the received limits to the robot controller.