To solve the problems of insufficient feature extraction, loss of detail texture information, and large number of model parameters in the current infrared and visible image fusion algorithms, a cross-scale pooling embedding image fusion algorithm with long- and short-distance attention collaboration is proposed. First, the depth separable convolution is used to design channel attention to enhance the expression of key channels and suppress redundant information. Second, based on group shuffle (GS) convolution, a multi-scale dense channel enhancement module is proposed, which enhances the multi-scale information interaction ability and reuses features by superimposing small convolution kernels and introducing dense connections to prevent information loss. Then, on the basis of the cross-scale embedding layer, a cross-scale pooling fusion embedding layer is proposed, and the features of some four stages are extracted using the fusion features of the pooling layer at different scales, so as to make full use of the features of each stage and reduce the computational complexity. Finally, the dual-path design is used to fuse long- and short-distance attention and design a convolutional feedforward network, so as to capture the dependence of long- and short-distance between features and reduce the amount of network parameters. Experimental results on the TNO and Roadscene public datasets of proposed algorithm and other seven algorithms show that, the outline of the fusion results by proposed algorithm is clear, and the entropy, average gradient, and structural content difference of proposed algorithm are improved compared with other algorithms, and the standard deviation of proposed algorithm is better on the Roadscence dataset. In addition, the detection performance comparison experiment of fused images on the M³FD dataset is carried out, and the experimental results show that the proposed algorithm performs well.