In one embodiment, the present invention includes a processor having multiple domains including at least a core domain and a non-core domain that is transparent to an operating system (OS). The non-core domain can be controlled by a driver. In turn, the processor further includes a memory interconnect to interconnect the core domain and the non-core domain to a memory coupled to the processor. Still further, a power controller, which may be within the processor, can control a frequency of the memory interconnect based on memory boundedness of a workload being executed on the non-core domain. Other embodiments are described and claimed.