The noise of repeaterless remote demodulation systems of fiber-optic hydrophone primarily comes from amplified spontaneous emission (ASE) noise and double Rayleigh scattering (DRS) noise. To optimize the noise and improve the performance of the system, the effects of ASE noise and DRS noise on phase noise of the system are separated experimentally by noise fitting. The results show that the ASE noise is the main noise source of the experimental system, while DRS noise also is the limiting factor of the phase noise reduction. The noise suppression effects of the illuminant phase modulation scheme on both ASE noise and DRS noise are theoretically analyzed and experimentally confirmed. Experimental results show that the total phase noise of 100 km remote demodulation system is suppressed by 5.2 dB when the sinusoidal phase modulation with amplitude of 2.2 rad and frequency of 40 MHz is applied by lithium niobate phase modulator, while ASE noise power is reduced by 5.2 dB and DRS noise power is reduced by 6.2 dB.