An optical communication system includes a digital signal processer coupled to the coherent receiver, said coherent receiver including a nonlinearity compensation module for compensating for nonlinear effects in fiber in the optical link for increasing capacity or transmission distance of the fiber, the nonlinearity compensation module includes a spectral slicing of the signal into bands, computing nonlinear interaction between the bands with parameters opposite to those of the fiber to reverse the non-linear effects in the fiber, and only certain nonlinear interactions between bands are considered thereby reducing complexity of the nonlinearity compensation.