A system and method relating to constructing an encoder and decoder neural network for providing semantic image segmentation includes generating an encoder comprising encoding convolution layers, each of the encoding convolution layers specifying an encoding filter operation using a respective first filter kernel, generating a decoder corresponding to the encoder, the decoder comprising decoding convolution layers, each of the decoding convolution layers being associated with a corresponding encoding convolution layer, and each of the decoding convolution layers specifying a decoding filter operation using a respective second filter kernel derived from the first filter kernel of the corresponding encoder convolution layer, and providing an input image to the encoder and the decoder for semantic image segmentation.