An HVAC controller that monitors a current performance level of an HVAC system, and determines if the current performance level falls below a minimum performance threshold level for at least a first predetermined amount of time. The HVAC controller may be configured it display an alert on the display of the HVAC controller if the HVAC controller determines that the current performance level has fallen below the minimum performance threshold level for at least the first period of time. In some cases, the HVAC controller may remove the alert from the display of the HVAC controller if the current performance level subsequently rises and remains above the minimum performance threshold level for at least a second predetermined amount of time. In some cases, the first predetermined amount of time may include a utility shutoff period component and/or a defrost cycle period component.