Systems and methods for automatic speech recognition by training a neural network to learn features from raw speech. The system comprises a neural network executing on a computer system and comprising a feature extractor, a label classifier, and a domain classifier. The feature extractor processes raw speech data and generates a first output data. The label classifier processes the first output data and generates a second output data. The domain classifier processes the first output data and generating a third output data. The neural network calculates first loss data based on the second output, and second loss data based on the third output. Further, the neural network is trained to minimize a cross-entropy cost of the label classifier and to maximize a cross-entropy cost of the domain classifier using the first loss data and the second loss data.