The wavelength demodulation of fiber Bragg grating laser sensor (FBGLS) system with 3×3 couplers depend on the physical characteristics of the 3×3 couplers. The demodulated results will distort to some extent due to the instability of the parameters in the three outputs of 3×3 couplers including the direct current (DC) component, the fringe visibility of interferometer and the phase difference. A method of measuring the above parameters is given and applied in the computer programming so as to measure the parameters in real-time under large signals, at the same time corresponding demodulation method is also introduced. The computer simulation shows that with the proposed method the appearance of harmonics caused by the instability of the parameters in the three outputs of 3×3 couplers is eliminated. The comparison of the experimentally demodulated results also shows that with the proposed method the harmonics have been reduced and the demodulated results have high correlation coefficients with the reference sensor. Besides, high resolution, dynamic range and linearity of the demodulated results are given in the experiment by using the proposed method.