The present approach relates to a system capable of life-long learning in a deep learning context. The system includes a deep learning network configured to process an input dataset and perform one or more tasks from among a first set of tasks. As an example, the deep learning network may be part of an imaging system, such as a medical imaging system, or may be used in industrial applications. The system further includes a learning unit communicatively coupled to the deep learning network 102 and configured to modify the deep learning network so as to enable it to perform one or more tasks in a second task list without losing the ability to perform the tasks from the first list.