A parallel access system including: a group of processing entities that comprises N processing entities; wherein N is a positive integer that exceeds one; a group of memory banks that stores K information elements; wherein the group of memory banks comprises N pairs of single access memory banks; each pair of memory banks comprises an even memory bank and an odd memory bank; wherein each pair of memory banks stores sub-set of K/N information elements; wherein an even memory bank of each pair of memory banks stores even address information elements of a certain sub-set of K/N information elements and an odd memory bank of each pair of memory banks stores odd address information elements of the certain sub-set of K/N information elements; wherein K/N is an even positive integer; and a non-blocking interconnect, coupled to the group of processing entities and to the group of memory banks; wherein during each fetch cycle each processing entity of the group of processing entities fetches a first information element from an odd memory unit of a pair of memory banks and fetches a second information element from an even memory unit of the pair of memory banks; wherein the first and second information elements are two consecutive interleaved address information elements.