A method for responding to inferred caller states during dialog with an intelligent voice interface configured to lead callers through pathways of an algorithmic dialog may include, during a voice communication with a caller via a caller device, receiving from the caller device caller input data indicative of a voice input of the caller, and determining, by processing the caller input data, an inferred state of the caller. Determining the inferred state of the caller may include analyzing one or more characteristics, other than textual content, of the voice input. The method may also include selecting a pathway through the algorithmic dialog based upon the inferred state of the caller.