Inspired by the contrast mechanism of the human visual system (HVS), this study proposed an improved high boost filter (IHBF)-based enhanced local contrast measurement method for solving the low detection rate of infrared (IR) small targets with a non-homogeneous background. First, based on the frequency characteristics of the small target, the IHBF operation was used to discard the low-frequency signal containing the background. An enhanced local contrast measure method was proposed to construct the contrast operator of the ratio-difference joint form. Thus, the target contrast can be enhanced further to obtain an optimal saliency map. Finally, the adaptive threshold technology was used to extract small targets. The simulation results demonstrate that compared with existing local contrast algorithms, the proposed method is better in terms of detection rate and false alarm rate and is an effective method for detecting IR small targets in non-homogeneous backgrounds.