The stability of the polarization encoder is one of the key issues affecting polarization coding communication. The stability of the polarization encoder based on lithium niobate (LN) phase modulator is studied in depth, and is described by time variant vector. The experiment results show that, for the LN phase modulator, the principal state of polarization-dependent loss (PDL) is consistent with the principal state of polarization-dependent phase shift, indicating that PDL can not affect the stability of the polarization state. A “inertia” of polarization rotation is observed in experiment. On the one hand, increasing voltage from 0 to a value, after stopping the increase of voltage, the polarization state can not be stable before about 30 min; on the other hand, decreasing voltage from a value to 0, after stopping the decrease of voltage, the polarization state can not be stable before several minutes too. It can have negative effects on low-speed modulation. For high-speed modulation, the changes of the average power can cause polarization jitter.