One illustrative device disclosed herein includes a layer of insulating material with its upper surface positioned at a first level and a recessed conductive interconnect structure positioned at least partially within the layer of insulating material, wherein a recessed upper surface of the recessed conductive interconnect structure is positioned at a second level that is below the first level. In this example, the device also includes a conductive cap layer positioned on the recessed upper surface of the recessed conductive interconnect structure, wherein an upper surface of the conductive cap layer is substantially co-planar with the upper surface of the layer of insulating material and a memory cell positioned above the conductive cap layer, wherein the memory cell comprises a lower conductive material that is conductively coupled to the conductive cap layer.