A distributed voice user interface system includes a local device which receives speech input issued from a user. Such speech input may specify a command or a request by the user. The local device performs preliminary processing of the speech input and also provides the speech input to a remote system. The local device is able to update its recognition capabilities based on analysis of the speech, input by the remote system.