A conductive memory stack is provided. The memory stack includes a bottom electrode, a top electrode and a multi-resistive state element. The multi-resistive state element is sandwiched between the electrodes such that the top face of the bottom electrode is in contact with the multi-resistive state element's bottom face and the bottom face of the top electrode is in contact with the multi-resistive state element's top face. The bottom electrode, the top electrode and the multi-resistive state element all have sides that are adjacent to their faces. Furthermore, the sides are at least partially covered by a sidewall layer.