Continuously monitoring heart rhythm can include grouping, using computer hardware, a plurality of inter-beat intervals (IBI) data for a user into a plurality of epochs, wherein each epoch includes a subset of the IBI data corresponding to a predetermined time span. For each epoch, a selected feature set selected from a plurality of feature sets is extracted based on a determination of temporal consistency of the epoch. A plurality of epoch classifications may be generated for the epochs using a selected feature processor, wherein each epoch classification indicates whether arrhythmia is detected for the epoch from which the epoch classification is generated. The selected feature processor is selected from a plurality of different feature processors on a per-epoch basis based on the selected feature set extracted from the epoch. An indication of arrhythmia may be output, via an output device of the computer hardware, based on the epoch classifications.