An apparatus for detecting an object in an image, including a detection group determiner configured to classify a plurality of feature levels into one or more detection groups, wherein the plurality of feature levels includes multiple scales and are arranged as a feature pyramid; and a detection group processor configured to create one or more feature maps corresponding to the one or more detection groups, and to detect an object by applying one or more models to the one or more feature maps.