Numerous embodiments of analog neural memory arrays are disclosed. In one embodiment, an analog neural memory system comprises an array of non-volatile memory cells, wherein the cells are arranged in rows and columns, the columns arranged in physically adjacent pairs of columns, wherein within each adjacent pair one column in the adjacent pair comprises cells storing W+ values and one column in the adjacent pair comprises cells storing W− values, wherein adjacent cells in the adjacent pair store a differential weight, W, according to the formula W=(W+)−(W−). In another embodiment, an analog neural memory system comprises a first array of non-volatile memory cells storing W+ values and a second array storing W− values.