In order to obtain an accurate model of the photoelectric tracking servo system, the identification using adaptive differential evolution algorithm based on the identification error sum was adopted. The mutation and crossover factor of the algorithm was automatically adjusted. The discrete model of system was identified under the situation of discretely sine digital signal input. Furthermore, the output of the first-order and second-order model using the sweeping frequency method in the frequency domain was compared with the output of the reality to prove the validity of the algorithm. Experimental result shows that under the same discretely sine signal input, the result of the identification is same with the real-life result and reduced about 20.33% in the RMSE compared with the sweeping frequency method. The second-order model has a bigger deviation in the high frequency domain and the first-order model has the same output with the real-life output. The adaptive differential evolution algorithm has a smaller amount of calculation and an accurate identification, besides, this method is simple enough. In summary, this method has a certain value in the engineering application.