A device can estimate the heart rate of an active user by using a physiological model to refine a “direct” measurement of the user's heart rate obtained using a pulse sensor. The physiological model can be based on heart rate response to activity and can be informed by context information, such as the user's current activity and/or intensity level as well as user-specific parameters such as age, gender, general fitness level, previous heart rate measurements, etc. The physiological model can be used to predict a heart rate, and the prediction can be used to assess or improve the direct measurement.