To address the problems of sparse distribution of small targets in infrared images is sparse, the proportion of pixels is small, and existing infrared small-target detection algorithms are vulnerable to strong noise interference, which significantly impact their accuracy and generalization, an infrared small-target detection algorithm based on context information fusion and visual saliency is proposed. First, the backbone network is constructed by an encoding-decoding method, in which the encoding layer is a full convolutional neural network stacked by hole convolution, and the input features are extracted. Then, feature fusion between different layers is realized through layer-by-layer skip splicing with the decoding layer, and feature information with strong semantics and strong location is extracted. Finally, the extracted features are input into the mixed domain module, and the channel attention and spatial attention mechanisms are used to improve the feature weight of small targets to enhance the background suppression. Through hole convolution combined with cross-layer fusion and visual saliency provided by the hybrid domain module, the proposed algorithm is demonstrated to be superior to the current typical algorithm for complex backgrounds. Compared with the comprehensive optimal algorithm, F_measure is improved by 10% on average, operation efficiency is increased by 40%, and detection and false alarm rate indicators are improved significantly.