Recently, multi-scale feature extraction has been widely used in the field of infrared and visible image fusion; however, most extraction processes are too complex, and the visual effect is not good. To improve the visual fidelity of the fusion result, a multi-scale horizontal image fusion model based on an edge-aware smoothing-sharpening filter (EASSF) is proposed for infrared and visible images. First, to obtain multiscale texture components and basic components in the horizontal direction, a multi-scale horizontal image decomposition method based on the EASSF is proposed to decompose the source image. Second, the maximum fusion rule is used to merge texture components, which can avoid loss of information detail. Then, to capture salient target information, the basic components are fused via the perceptual-fusion rule. Finally, the fused image is obtained by integrating the fused multi-scale texture components and basic components. By analyzing the perceptual fusion coefficient of PF, the appropriate range of infrared and visible image fusion in the multi-scale EASSF is obtained through the objective data of the fusion results. In this range, compared with several classical and popular fusion methods, the proposed fusion model not only avoids the complexity of feature information extraction, but also effectively ensures the visual fidelity of fusion results by integrating the significant spectral information of basic components.