Systems and processes for operating an intelligent automated assistant are provided. In one example process, discourse input representing a user request can be received. The process can determine one or more possible device characteristics corresponding to the discourse input. Data structure representing a set of devices of an established location can be retrieved. The process can determine, based on the data structure, one or more candidate devices from the set of devices. The one or more candidate devices can correspond to the discourse input. The process can determine, based on the one or more possible device characteristics and one or more actual device characteristics of the one or more candidate devices, a user intent corresponding to the discourse input. Instructions that cause a device of the one or more candidate devices to perform an action corresponding to the user intent can be provided.