Aspects of the disclosure are directed to a canonical approach for feature selection referred to as sparse learnable masks (SLM). SLM integrates learnable sparse masks into end-to-end training. For the fundamental non-differentiability challenge of selecting a desired number of features, SLM includes dual mechanisms for automatic mask scaling by achieving a desired feature sparsity and gradually tempering this sparsity for effective learning. SLM further employs an objective that increases mutual information (MI) between selected features and labels in an efficient and scalable manner. Empirically, SLM can achieve or improve upon state-of-the-art results on several benchmark datasets, often by a significant margin, while reducing computational complexity and cost.