Target tracking plays an important role in the military, medical and other fields, and Field Programmable Gate Array (FPGA) is widely used in the direction due to its good performance and high flexibility. However, at present, limited to the complexity of high-precision tracking algorithms, most of the target tracking systems are implemented by foreign high-performance chips, which leads to weak autonomy and controllability. If domestic chips are used to achieve target tracking, it will face the problem that there are few IP cores and most modules need to be designed in Verilog. In addition, the feasibility of the research algorithm in other domestic FPGAs needs to be considered. Therefore, the objective is to study a tracking algorithm that is easy to design in Verilog, has generalization, and improves real-time and robustness.Template matching is easy to design with pipelines and is selected as the basic algorithm, which is widely used due to its simplicity and accuracy. Among them, the template matching algorithm based on Sum of Absolute Difference (SAD) has no multiplication and division operation, which is suitable for FPGA implementation with limited resources. This tracking algorithm has too strict constraints, which leads to the problem of insufficient robustness. Based on the Sum of Absolute Difference (SAD) similarity measurement method, a method for finding the Sum of Minimum Absolute Difference (SMAD) in the window is proposed. In order to reduce resource usage, the maximum and minimum filtering (Fig.3) is used to preprocess the image data in the window and then the minimum absolute difference is obtained, which reduces the resource consumption of the addition and subtraction of the SMAD method to 31.8%. Moreover, a pyramid-like update strategy (Fig.4) that is easy to implement by FPGA hardware is proposed to better adapt to the scale change of the target. In order to verify the tracking performance of proposed algorithm, Unigroup FPGAs are used to implement it and build a real-time target tracking system (Fig.5).Based on the two indicators of Average Overlap Rate (AOR) and success rate, the algorithm comparison experiment was carried out with the OTB dataset. It verified that the proposed algorithm has certain anti-occlusion and scale adaptability (Fig.10). Compared with the SAD method, the tracking metrics in each scenario are improved. In the scale change and synthesis scenario, after the SMAD method is added to the pyramid-like strategy, its success rate and AOR are improved, which verifies the effectiveness of the pyramid-like update strategy (Tab.2). Compared with the robust DDIS algorithm, the proposed method improves the average success rate and overlap rate by 1.18% and 0.13%, respectively, and is easier to design with FPGA. Then the target tracking system is implemented by domestic FPGAs. The delay time is 16 line synchronization cycles plus 37 clock cycles and tracking frame rate can reach 100 frames per second. Different outdoor backgrounds were selected to test the tracking of the system when the target changed in scale, direction of motion and speed (Fig.12). The anti-occlusion test experiment of the tracking platform shows that (Fig.13) when the target is partially occluded, the tracking system can still successfully track the target, which further verifies the feasibility of the proposed method.In order to break the technology monopoly and improve the autonomy of target tracking application products, the traditional SAD template matching method is improved considering the limitations of domestic FPGA and the performance of tracking algorithms. The SMAD method is proposed and its resource consumption is optimized. Combined with a pyramid-like template update strategy, its tracking performance is improved. The experiments of OTB dataset and domestic tracking system verify its tracking effect. It provides a reference scheme for the localization of high frame rate target tracking system.