As one of the important directions of computer vision, target tracking has a wide range of applications, such as autopilot, UAV tracking, but the target tracking algorithm cannot run effectively on embedded devices. A novel acceleration target tracking scheme based on correlation filtering was proposed to solve the problems of target tracking algorithm, such as high computation and complexity, difficulty application on the resource-constrained embedded devices. Firstly, the adaptive fuzzy algorithm was used to optimize the overall computation of the algorithm, which could decide whether to reduce the image quality based on target size. Secondly, the criterion of Peak-to-Sidelobe Rate and Average Peak-to-Correlation Energy were used to measure the reliability of tracking results, so as to realize adaptive updating of tracking model and re-search of target location. Finally, for the correlation operation and complex matrix multiplication operation in the stage of training tracking detector, which were implemented based on FPGA parallelly to improve the real-time energy efficiency of the algorithm. The proposed acceleration algorithm was deployed on PYNQ-Z2 and verified based on OTB-2015 tracking data set. The tracking accuracy and real-time performance of the algorithm were 65.8% and 17.28 frame/s, respectively, compared with the original algorithm, the tracking accuracy and real-time performance were improved by 9.12% and 703.7%, respectively.