The disclosed structures and methods are directed to a method for compensation of linear and nonlinear effects in optical fiber of a coherent optical signal transmitted through an optical link. The method comprises receiving a coherent optical signal having carriers; determining values of intensity vectors for each carrier; determining values of filtered intensity vectors for each carrier by filtering the values of the intensity vectors at frequencies lower than a cut-off frequency of a filter; determining nonlinear compensation coefficients for each carrier based on the filtered intensity vectors; and modifying the digital coherent optical signal based on the nonlinear compensation coefficients.