  In this paper, the performance of adaptive turbo equalization for
nonlinearity compensation (NLC) is investigated. A turbo equalization scheme is
proposed where a recursive least-squares (RLS) algorithm is used as an adaptive
channel estimator to track the time-varying intersymbol interference (ISI)
coefficients associated with inter-channel nonlinear interference (NLI) model.
The estimated channel coefficients are used by a MIMO 2x2 soft-input
soft-output (SISO) linear minimum mean square error (LMMSE) equalizer to
compensate for the time-varying ISI. The SISO LMMSE equalizer and the SISO
forward error correction (FEC) decoder exchange extrinsic information in every
turbo iteration, allowing the receiver to improve the performance of the
channel estimation and the equalization, achieving lower bit-error-rate (BER)
values. The proposed scheme is investigated for polarization multiplexed 64QAM
and 256QAM, although it applies to any proper modulation format. Extensive
numerical results are presented. It is shown that the scheme allows up to 0.7
dB extra gain in effectively received signal-to-noise ratio (SNR) and up to 0.2
bits/symbol/pol in generalized mutual information (GMI), on top of the gain
provided by single-channel digital backpropagation.
