The present disclosure generally relates to systems and methods for processing received voice inputs for user identification. In an example process, voice input can be processed using a subset of words from a library used to identify the words or phrases of the voice input. The subset can be selected such that voice inputs provided by the user are more likely to include words from the subset. The subset of the library can be selected using any suitable approach, including for example based on the user's interests and words that relate to those interests. For example, the subset can include one or more words related to media items stored by the user on the electronic device, names of the user's contacts, applications or processes used by the user, or any other words relating to the user's interactions with the device.