The characteristics of the speaker may be used to automatically customize a user interface of a client device for the speaker. For instance, a mobile device may generate a user interface that does not permit access to particular applications on the mobile device, that only includes options to call home and access a camera application, or that includes an override option to cause generation of a user interface for a different class of user than a user class for the automatic customizations applied to the user interface.