The digital quadrature demodulation algorithm of phase sensitive optical time domain reflectometer is investigated. The influence of the group delay of the digital filter in quadrature demodulation algorithm on the positioning accuracy is analyzed, and the relocation method is proposed to correct the positioning error. Meanwhile, the influences of the digital filter in the quadrature demodulation algorithm on the spatial resolution of the system is simulated and experimented. The results show that the relocation method can eliminate the positioning error caused by the digital filter in the quadrature demodulation algorithm. The spatial resolution of the system is determined by the system structure and the digital filter, and the spatial resolution of the digital filter improves with the increase of the order of the digital filter. In this experiment, the demodulation is appropriate to select 112 order filter.