In the polarization mode dispersion(PMD) compensation experiment,the degree of polarization (DOP) is used as the most common feedback signal,which reflects PMD and its influence on the transmission system factually. The influences of first-order and second-order PMD on DOP of four optical modulation formats(RZ,CSRZ,RZ-DPSK and CSRZ-DPSK) are compared and mathematically analyzed.The result shows that different formats have different DOP and the second-order PMD has less influence on DOP of CSRZ-DPSK format,which is fitter for mitigating PMD than the other formats in self-adaptive PMD compensation system.