A hierarchical on-chip memory (400) includes an area distributed CMOS layer (310) comprising input/output functionality and volatile memory and via array (325, 330), the area distributed CMOS layer (310) configured to selectively address the via array (325, 330). A crossbar memory (305) overlies the area distributed CMOS layer (310) and includes programmable crosspoint devices (315) which are uniquely accessed through the via array (325, 330). A method for utilizing hierarchical on-chip memory (400) includes storing frequently rewritten data in a volatile memory and storing data which is not frequently rewritten in a non-volatile memory (305), where the volatile memory is contained within an area distributed CMOS layer (310) and the non-volatile memory (305) is formed over and accessed through the area distributed CMOS layer (310).