The present approach relates to feature ranking within deep neural networks in a multi-task and/or multi-label setting. Approaches are described to identify features that are task-specific as well as features that are shared across multiple tasks. In addition to facilitating interpretability, the selected subset of features can be used to make efficient models leading to better stability & regularization along with reduced compute and memory.