In accordance with one embodiment, an electrochemical cell includes a first anode including a form of lithium a first cathode including an electrolyte, and a first composite electrolyte structure positioned between the first anode and the first cathode, the first composite electrolyte structure including (i) a first support layer adjacent the first anode and configured to mechanically suppress roughening of the form of lithium in the first anode, and (ii) a first protective layer positioned between the first support layer and the first cathode and configured to prevent oxidation of the first support layer by substances in the first cathode.