An apparatus for recognizing multi-user interactions includes: a pre-processing unit for receiving a single visible light image to perform pre-processing; a motion region detecting unit for detecting a motion region from the image to generate motion blob information; a skin region detecting unit for extracting information on a skin color region from the image to generate a skin blob list; a Haar-like detecting unit for performing Haar-like face and eye detection by using only contrast information from the image; a face tracking unit for recognizing a face of a user from the image by using the skin blob list and results of the Haar-like face and eye detection; and a hand tracking unit for recognizing a hand region of the user from the image.