A cached memory system that can handle high-rate input data and ensure that an embedded DSP can meet real-time constraints is described. The cached memory system includes a cache memory located close to a processor core, an on-chip memory at the next higher memory level, and an external main memory at the topmost memory level. A cache controller handles paging of instructions and data between the cache memory and the on-chip memory for cache misses. A direct memory exchange (DME) controller handles user-controlled paging between the on-chip memory and the external memory. A user/programmer can arrange to have the instructions and data required by the processor core to be present in the on-chip memory well in advance of when they are actually needed by the processor core.