The multi-pulse position modulation combined with quadrature phase shift keying (MPPM-QPSK) has obvious advantages in receiver sensitivity. Equalization algorithms can effectively cope with the influence of limited signal bandwidth and inter-symbol interference in digital coherent optical communication systems and improve the signal transmission quality. To tackle the problem that only the QPSK part of MPPM-QPSK can be optimized by the constant modulus algorithm, this paper improves the inner-ring reference modulus of the two modulus equalization algorithm and proposes a two modulus equalization algorithm with the inner-ring modulus not 0. The algorithm is applied to the MPPM-QPSK coherent optical communication system of 10 Gbit/s single-carrier Gaussian shaping and Nyquist shaping, and some parameters are optimized, such as the inner-ring modulus, the number of taps and the step length. The experimental results show that the receiver sensitivity of the system is optimized by about 0.1 dB compared with that of the system using the traditional two modulus algorithm. When the number of pulses is set at m=2, 4, 8, 16, the receiver sensitivity of this system is improved by 0.9 dB, 0.6 dB, 0.5 dB and 0.4 dB, respectively, in comparison with the system without equalization algorithm.