Numerous embodiments of analog neural memory arrays are disclosed. In one embodiment, a system comprises a first array of non-volatile memory cells, wherein the cells are arranged in rows and columns and the non-volatile memory cells in one or more of the columns stores W+ values, and wherein one of the columns in the first array is a dummy column; and a second array of non-volatile memory cells, wherein the cells are arranged in rows and columns and the non-volatile memory cells in one or more of the columns stores W− values, and wherein one of the columns in the second array is a dummy column; wherein pairs of cells from the first array and the second array store a differential weight, W, according to the formula W=(W+)−(W−).