Systems and processes for modifying word predictions are provided. In one example, a user input is received including one or more words. A prediction of a word sequence corresponding to one or more words is obtained, and context information associated with the word sequence is obtained. In accordance with a determination, based on the context information, that the prediction of the word sequence corresponds to a predetermined semantic reference, the prediction of the word sequence is modified, and an output is provided corresponding to the modified prediction of the word sequence. In accordance with a determination, based on the context information, that the prediction of the word sequence does not correspond to a predetermined semantic reference, an output is provided corresponding to the prediction of the word sequence.