The algorithm based on multi-scale mathematical morphology fusion filter is presented to resolve the infrared and TV image detection polluted by heavy yawp in the shipboard infrared search and track system. First the target image is filtered through multi-scale mathematical morphology operators, bigger morphology operator eliminating noise and smaller morphology operator picking up image detail. The simultaneity transacted results are fused based on wavelet transform. Across resolution evaluation operator of target and background is presented to solve the performance of match filtering arithmetic. The experimental results show that the algorithm can eliminate the strong clutter of clouds, ocean waves, sea level and strong noise and is better than median and wavelet transform algorithm. The experimental results also show that the quantative evaluation algorithm of filtered image reasonable and effective. The algorithmis suited for shipboard infrared surveillance system.