In this paper, an infrared-image-based algorithm is proposed for the detection of dim and small targets in complex backgrounds. The proposed algorithm is based on the contrast mechanism of the human visual system. First, an infrared image was preprocessed, and isolated noise points in the image were removed via median filtering. The processed image was then subjected to difference-of-Gaussians filtering to suppress large-area highlighted areas in the image. Finally, an improved local contrast algorithm was used to remove the highlighted edge regions and eliminate the high suspect target to achieve the detection of dim and small targets in complex backgrounds using infrared images. Experimental results show that compared with the traditional LCM algorithm, top-hat algorithm, TDLMS algorithm, and infrared patch-image model, the proposed algorithm is more advantageous with regard to the false alarm rate, correct detection rate, detection time, etc. It also has the characteristics of a high detection rate, low false alarm rate, good robustness, and short running time.