This work presents a method based on contrast and structure extraction to overcome the the difficulties of low contrast and unclear edge contour in the fusion of infrared and visible images caused by existing multi-scale transformation methods. First, the visible image is adaptively improved and the infrared image is linearly normalized. Then, the local contrast and salient structure of the image are extracted using dense SIFT descriptors and local gradient energy operators, respectively, and the weight map is obtained by combining the weights of the local contrast and salient structure. The weight map eliminates discontinuities and noise with a rapid guidance filter. Finally, the weight map after thinning and the source image after enhancement and linear normalization are fused by the pyramid decomposition technique. Moreover, this research conducted a large number of experiments on publicly available datasets using six evaluation indicators to quantitatively examine the experimental outcomes, and qualitatively compared the proposed method with 10 mainstream image fusion algorithms. The experimental findings show that the suggested method can effectively preserve the contrast, edge contour, and detail information of the source image while achieving the best fusion effect in visual perception and quantitative indicators.