A processor includes a register to store an encoded pointer to a memory location in memory and the encoded pointer is to include an encrypted portion. The processor further includes circuitry to determine a first data encryption factor based on a first data access instruction, decode the encoded pointer to obtain a memory address of the memory location, use the memory address to access an encrypted first data element, and decrypt the encrypted first data element using a cryptographic algorithm with first inputs to generate a decrypted first data element. The first inputs include the first data encryption factor based on the first data access instruction and a second data encryption factor from the encoded pointer.