There is disclosed a portable security device for securing a data exchange between a host device and a remote device, said portable security device comprising a processing unit, a secure element and a data interface, wherein: the secure element is arranged to store an encryption key and a decryption key; the processing unit is arranged to control the encryption of data to be transmitted from the host device to the remote device, wherein said encryption is performed using said encryption key; the processing unit is further arranged to control the decryption of data transmitted from the remote device to the host device, wherein said decryption is performed using said decryption key. Furthermore, a corresponding method for securing a data exchange between a host device and a remote device using a portable security device is disclosed, as well as a corresponding computer program product.