The invention relates to a reactor melt out-of-core retention system comprising a collection and retention device arranged at the bottom of a reactor pit of a reactor; the collection and retention device and the reactor pit of the reactor communicate via a melting plug arranged at the top of the collection and retention device; the collection and retention device is internally provided with an expansion and cooling subsystem; the expansion and cooling subsystem includes an expansion and cooling subsystem main body; the expansion and cooling subsystem main body is a plate-like spiral body, a plurality of cooling pipes are arranged inside the expansion and cooling subsystem main body, two ends of the plurality of cooling pipes are respectively connected with a water delivery inlet pipeline of the expansion and cooling subsystem and a water delivery outlet pipeline of the expansion and cooling subsystem, and the water delivery inlet pipeline of the expansion and cooling subsystem and the water delivery outlet pipeline of the expansion and cooling subsystem communicate through a condenser. When a serious accident of reactor core meltdown happens in a nuclear power plant, the out-of-core retention system can rapidly and effectively expand and cool reactor core melts in a small space to a greatest extent, so as to reduce a reactor pit melt-through risk and a radioactive material release risk, and improve the safety of the reactor accidents.