Delivering a Direct Proof private key to a device installed in a client computer system in the field may be accomplished in a secure manner without requiring significant non-volatile storage in the device. A unique pseudo-random value is generated and stored in the device at manufacturing time. The pseudo-random value is used to generate a symmetric key for encrypting a data structure holding a Direct Proof private key and a private key digest associated with the device. The resulting-encrypted data structure is stored on a removable storage medium (such as a CD), and distributed to the owner of the client computer system. When the device is initialized on the client computer system, the system checks if a localized encrypted data structure is present in the system. If not, the system obtains the associated encrypted data structure from the removable storage medium. The device decrypts the encrypted data structure using a symmetric key regenerated from its stored pseudo-random value to obtain the Direct Proof private key. If the private key is valid, it may be used for subsequent authentication processing by the device in the client computer system.