Methods and apparatus for extending packet processing to trusted programmable and fixed-function accelerators. Secure enclaves are created in system memory of a compute platform, wherein software code external from a secure enclave cannot access code or data within a secure enclave, and software code in a secure enclave can access code and data both within the secure enclave and external to the secure enclave. Software code for implementing packet processing operations is installed in the secure enclaves. The compute platform further includes one or more hardware-based accelerators that are used by the software to offload packet processing operations. The accelerators are configured to read packet data from input queues, process the data, and output processed data to output queues, wherein the input and output queues are located in encrypted portions of memory that may be in a secure enclave or external to the secure enclaves. Tokens are used by accelerators to validate access to memory in secure enclaves, and used by both accelerators and secure enclaves to access encrypted memory external to secure enclaves.