Techniques are provided to improve the performance and accuracy of landmark point detection using a Constrained Local Model. The accuracy of feature filters used by the model may be improved by supplying positive and negative sets of image data from training image regions of varying shapes and sizes to a linear support vector machine training algorithm. The size and shape of regions within which a feature filter is to be applied may be determined based on a variance in training image data for a landmark point with which the feature filter is associated. A sample image may be normalized and a confidence map generated for each landmark point by applying the feature filters as a convolution on the normalized image. A vector flow map may be pre-computed to improve the efficiency with which a mean landmark point is adjusted toward a corresponding landmark point in a sample image.