Systems and methods for self-supervised representation learning as a means to generate context-specific pretrained models include selecting data from a set of available data sets; selecting a pretext task from domain specific pretext tasks; selecting a target problem specific network architecture based on a user selection from available choices or any customized model as per user preference; and generating a pretrained model for the selected network architecture using the selected data obtained from the set of available data sets and a pretext task as obtained from domain specific pretext tasks.