Disclosed herein are a spalling-preventing composite material composed of fiber and powder, which have different diameters and melting points so as to be capable of realizing the effect of preventing spalling of high-strength concrete and the effect of improving the fluidity of concrete, and a high-strength refractory concrete comprising the spalling-preventing material. The composite material for preventing spalling of high-strength concrete is composed of powder and fiber at 1:1-3, wherein the powder is a polymer powder having a diameter of 0.10-0.5 mm and a melting point of 110-150° C., and the fiber is a conjugate fiber including a first fiber having a diameter of 0.05-0.10 mm, a length of 5-25 mm and a melting point of 150-190° C., and a second fiber having a diameter of 0.01-0.05 mm, a length of 5-25 mm and a melting point of 190-250° C., the first fiber being a polypropylene fiber, and the second fiber being a nylon fiber or a polyvinyl alcohol fiber.