The present disclosure provides a graphics processing unit, GPU, comprising: a plurality of memory controllers, a cache memory coupled with the plurality of memory controllers, a a graphics multiprocessor coupled with the cache memory and the plurality of memory controllers. The graphics multiprocessor having a single instruction, multiple thread, SIMT, architecture. The graphics multiprocessor includes a register file and a plurality of compute units coupled with the register file. The plurality of compute units including a first compute unit to perform a mixed precision matrix operation and a second compute unit to perform, in response to a single instruction, multiple compute operations, wherein the multiple compute operations include a fused multiply-add operation and a rectified linear unit operation applied to an output of the fused multiply-add operation.