A security system includes biometric authentication such as voice authentication with wireless tag detection to identify both the user and a wireless tag carried by the user. When the user approaches the entrance to a building that is secured by the security system, a tag detector of a user interface receives an identifier from the tag. The user interface transmits the tag identifier to a base station for comparison to pre-stored tag identifiers in a database. If there is a match, the user is prompted to provide a biometric input, such as by speaking a pre-arranged phrase. The biometric input is compared with templates stored in the database to determine if there is a match. If there is a match, the security system is disarmed. The template can be selected based on the tag identifier to improve processing speed.