A method of decoding multi-layer video includes decoding a first layer picture, storing the decoded first layer picture in a decoded picture buffer (DPB), and marking the first layer picture as a short-term reference picture; obtaining interlayer reference picture set (RPS) information regarding a second layer picture having a same first picture order count (POC) as the first layer picture; marking the first layer picture, which is marked as the short-term reference picture, as a long-term reference picture, based on the interlayer RPS information; performing interlayer prediction with respect to the second layer picture with reference to the first layer picture; decoding the second layer picture based on a result of performing the interlayer prediction; and, after the second layer picture is decoded, marking the first layer picture, which is marked as the long-term reference picture, as a short-term reference picture.