Detection of a facial attribute such as a smile or gender in a human face in an image is performed by embodiments of the present invention in a computationally efficient manner. First, a face in the image is detected to produce a facial image. Facial landmarks are detected in the facial image. The facial image is aligned and normalized based on the detected facial landmarks to produce a normalized facial image. Local features from selected local regions are extracted from the normalized facial image. A facial attribute is predicted in each selected local region by inputting each selected local feature into a weak classifier having a multi-layer perceptron (MLP) structure. Finally, output data is aggregated from each weak classifier component to generate all indication that the facial attribute is detected in the facial image.