Disclosed is neural network circuitry having a first plurality of logic cells that is interconnected to form neural network computation units that are configured to perform approximate computations. The neural network circuitry further includes a second plurality of logic cells that is interconnected to form a controller hierarchy that is interfaced with the neural network computation units to control pipelining of the approximate computations performed by the neural network computational units. In some embodiments the neural network computation units include approximate multipliers that are configured to perform approximate multiplications that comprise the approximate computations. The approximate multipliers include preprocessing units that reduce latency while maintaining accuracy.