The tunable grating filtering detection system of fiber grating sensor is investigated theoretically and experimentally. The theoretical expression of the wavelength resolution is presented. The optimum theoretical wavelength resolution of the system is 0.8 pm. Due to the excellent performance of the photoelectric detection system, the measured wavelength resolution of sensing detection is 2 pm and the strain resolution is 1.7 με in the experiment.