Aiming at the influence of hardware noise and sea background clutter in the infrared images of marine targets, we analyze the noise of offshore infrared target tracking working systems based on the background of an offshore unmanned vehicle video navigation obstacle avoidance system, and design a related denoising algorithm. This algorithm completes the system modeling and improves the parameters. Then, the infrared video at sea is sampled frame by frame, and the sampled infrared image sequence is filtered. The tracking position before and after filtering is compared with the central coordinate of the target real position. Finally, the processing result of this algorithm is compared with other similar algorithms. Experimental results show that using the proposed algorithm, the tracking position after filtering basically coincides with the real position, the capture rate is higher than 98%, and the tracking error is less than 1 mrad. Compared with similar filtering algorithms, the signal-to-noise ratio is improved by 10 dB.