Because the signal light is very weak and the communication link is easily interfered, space optical communication and space quantum communication require extremely high precision for the tracking system of the terminal payload. The accuracy of the fine tracking subsystem determines the tracking accuracy of the entire terminal system. Therefore, in the fine tracking stage, the system must meet the requirements of high precision and large bandwidth. However, the accuracy of traditional fine tracking systems is easily affected by external disturbances. To achieve higher tracking accuracy and stronger interference suppression capability, based on the traditional precise tracking system of typical optical communication terminals, a design method of an additional integrated module is proposed, and this module is cascaded after the PID controller of the traditional control system. Based on performance indicators such as control bandwidth, interference suppression capability, and stability, the non-dominated sorting genetic algorithm II is used to obtain the global optimal controller parameters, and a precise tracking system with intelligent parameter search is realized, which can achieve the tracking accuracy of sub-micro radian scale. Based on the measured angular interference data of a typical optical communication satellite terminal in orbit, the simulation compares the new system and the traditional system. The results show that on the basis of maintaining the stability of the closed-loop system, the new system can increase the error suppression bandwidth by 33.7% and improve the interference suppression ability of the full frequency band by 19.5%, of which the interference error suppression performance within 10 Hz is improved by more than 95%. For the four frequency bands of 0~1 Hz, 1~10 Hz, 10~50 Hz, and 50~100 Hz, the accuracy of the new system is improved by 99.5%, 95.7%, 71.3%, and 29.9% respectively compared with the traditional system. A physical verification system is built in the laboratory environment, and it is verified that the tracking accuracy and interference suppression performance of the system are greatly improved compared with the traditional system, especially below 10 Hz, the improvement rate is more than 20 times. When the interference frequency is 5 Hz and 10 Hz, the interference rejection ratio of the system reaches -53.57 dB and -46.31 dB, respectively. The experimental results and simulation results are consistent with a good fit. The system can be used in space optical communication scenarios with longer distances and higher precision requirements, which is of great significance to the development of the future space optical communication field.