Embodiments relate to a human behavior recognition system using hierarchical class learning considering safety, the human behavior recognition system including a behavior class definer configured to form a plurality of behavior classes by sub-setting a plurality of images each including a subject according to pre-designated behaviors and assign a behavior label to the plurality of images, a safety class definer configured to calculate a safety index for the plurality of images, form a plurality of safety classes by sub-setting the plurality of images based on the safety index, and additionally assign a safety label to the plurality of images, and a trainer configured to train a human recognition model by using the plurality of images defined as hierarchical classes by assigning the behavior label and the safety label as training images.