To reduce the interference caused by channel noise and polarization mode dispersion (PMD) in the optical communication, a modified forward error correction method is studied. By inserting fixed bits which are defined as the watermark bits (water-bit ) into the low density parity check (LDPC) codes to estimate the channel, a new decoding algorithm which is combined with channel estimation is proposed. The results show that the LDPC code with 16、 32 water-bit transmitted over Gaussian channel outperforms the conventional LDPC code transmitted over Chi-square channel in the error correction performance, and with bit error rate (BER) below 10-9 as standard, the optical signal to noise ratio (OSNR) value can reduce about 0.25 dB、 0.35 dB. When PMD is considered in the high-speed optical communications, the LDPC code with 16、 32 water-bit has a better tolerance than the conventional LDPC code, when differential group delay (DGD) is 20 ps and 30 ps, the OSNR value can reduce about 0.63 dB、 0.98 dB and 1.16 dB、 1.87 dB.