An image-processing apparatus for executing accurate facial expression recognition even for a subject hard to recognize a facial expression is provided. A person's face region is extracted from an image input from an image input unit. A predetermined partial region that changes between when the facial expression is in the first and second states is extracted from the extracted face region. A facial expression evaluation value is calculated using an evaluation value calculation formula. When the calculated facial expression evaluation value exceeds a threshold value, it is determined that the facial expression is in the second state. If the difference between the maximum value and the minimum value of the calculated facial expression evaluation value within a predetermined time is smaller than a predetermined value, the evaluation value calculation formula or its parameter is changed to increase the difference.