Current technologies analyze electrocardiogram (ECG) signals for a long duration, which is not always a practical scenario. Moreover the current scenarios perform a binary classification between normal and Atrial Fibrillation (AF) only, whereas there are many abnormal rhythms apart from AF. Conventional systems/methods have their own limitations and may tend to misclassify ECG signals, thereby resulting in an unbalanced multi-label classification problem. Embodiments of the present disclosure provide systems and methods that are robust and more efficient for classifying rhythms for example, normal, AF, other abnormal rhythms and noisy ECG recordings by implementing a spectrogram based noise removal that obtains clean ECG signal from an acquired single-lead ECG signal, an optimum feature selection at each layer of classification that selects optimum features from a pool of extracted features, and a multi-layer cascaded binary classifier that identifies rhythms in the clean ECG signal at each layer of the classifier.