The fundamental of two stage modulation in intensity modulation direct detection (IM-DD) high-speed dense wavelength-division multiplexing (DWDM) systems is depicted. And based on the transfer characteristic of Mach-Zehnder modulator, the mathematical expressions of four modulation formats are derived including: full frequency modulated return-to-zero (FFMRZ), half frequency modulated return-to-zero (HFMRZ), single sideband modulated return-to-zero (SSBRZ) and carrier suppressed return-to-zero (CSRZ). The characteristics of both time domain and frequency domain are represented according to optical eye diagrams and normalized spectrum. Then the dispersion limit and nonlinearity limit are numerically investigated respectively by comparing the eye opening penalty (EOP), and CSRZ performs better than other formats. At last, the Q-factor, which is relative to the system bit error ratio (BER), is obtained via numerical simulation utilizing comprehensive DWDM system model. The performance of CSRZ still maintains the best and the superiority is more evident when polarization mode dispersion (PMD) is considered. In conclusion, CSRZ is a preferable choice in IM-DD systems.