A controller offers various security modes for protecting program code and data stored in memory and ensuring that the protection is effective during all normal operating conditions of the controller. The controller includes configuration settings that segment program memory into a boot segment, a secure segment and a general segment, each with a particular level of security including no enhanced protection. The boot code segment (BS) is the most secure and may be used to store a secure boot loader. The secure code segment (SS) is useful for storing proprietary algorithms from third parties, such as algorithms for separating ambient noise from speech in speech recognition applications. The general code segment (GS) has the least security. The controller is configured to prevent program flow changes that would result in program code stored in high security segments from being accessed by program code stored in lower security segments. In addition, the processor may be configured to have associated secure data portions of both program memory, such as flash memory, and random access memory (RAM) corresponding to the BS, SS and GS. Attempts to read data from or write data to the program memory or RAM associated with a higher security level from a lower security level are prevented from occurring.