To reduce influence of quadrant detector output signal nonlinear error on the system precision, a matched condition was given to ensure maximum linear interval of detector output signal, and characteristic optimization of detector output signal by subsection linearization method was put forward. In the paper, a mathematical model of quadrant detector output signal was established by light energy integral. Based on the model, the relations were analyzed between linear characteristic of signal with radius ratio of spot to photosensitive area, signal sampling precision and non-uniform of photoelectric transform coefficient and so on. Then, matched condition was acquired which is most proper for radius ratio of spot to photosensitive area corresponding to maximum linear interval of detector’ output signal. And the optimum linearity characteristic parameters of detector’ signal were gotten by subsection linearization fitting. The simulation result shows that, compared with multi-calibration and least square fitting in entire linear interval methods, max and mean of nonlinear errors in entire linear interval are nearly reduced to 1/10 of the method’s mentioned previously, and accordingly, the linear interval of detector is expanded 20%~30% under the same condition.