A biometric identity broker can obtain exclusive control of biometric sensors communicationally coupled to a computing device, and can create biometric unit components for each of the sensors. Each biometric unit component can utilize a sensor plug-in, a processing plug-in and a storage plug-in, that can either be provided with the sensor or can be default plug-ins, in order to provide a baseline level of functionality irrespective of the functionality provided by, or with, the sensor to which the biometric unit is associated. A factor-specific service provider can act across all biometric units associated with sensors directed to a common biometric factor and can arbitrate among multiple clients simultaneously awaiting input and can manage multiple identities associated with a single biometric template based on the sensor receiving the biometric input. The biometric identity broker, factor-specific service providers and biometric units can be part of the trusted computing base.