Systems and processes for operating an intelligent automated assistant are provided. An example method includes, at an electronic device having one or more processors and memory: receiving a natural language speech input; determining, based on the natural language speech input, a plurality of candidate intents; obtaining contextual data associated with the user device; ranking, based on the contextual data, the plurality of candidate intents using a machine learning model, wherein the machine learning model is pre-trained at least partially on the user device; determining a user intent based on the ranked candidate intents; and performing a task corresponding to the determined user intent.