Remote Sensing Image Fusion Based on Two-branch U-shaped Transformer

Wensheng FAN; Fan LIU; Ming LI

doi:10.3788/gzxb20235204.0428002

[1] P GHAMISI, B RASTI, N YOKOYA et al. Multisource and multitemporal data fusion in remote sensing: a comprehensive review of the state of the art. IEEE Geoscience and Remote Sensing Magazine, 7, 6-39(2019).

[2] F BOVOLO, L BRUZZONE, L CAPOBIANCO et al. Analysis of the effects of pansharpening in change detection on vhr images. IEEE Geoscience and Remote Sensing Letters, 7, 53-57(2010).

[3] J GILBERTSON, J KEMP, A NIEKERK. Effect of pan-sharpening multi-temporal landsat 8 imagery for crop type differentiation using different classification techniques. Computers and Electronics in Agriculture, 134, 151-159(2017).

[4] G VIVONE, M MURA, A GARZELLI et al. A new benchmark based on recent advances in multispectral pansharpening: revisiting pansharpening with classical and emerging pansharpening methods. IEEE Geoscience and Remote Sensing Magazine, 9, 53-81(2021).

[5] W CARPER, T LILLESAND, R KIEFER. The use of intensity-hue-saturation transformations for merging spot panchromatic and multi-spectral image data. Photogrammetric Engineering and Remote Sensing, 56, 459-467(1990).

[6] P KWARTENG, A CHAVEZ. Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis. Photogrammetric Engineering and Remote Sensing, 55, 339-348(1989).

[7] B AIAZZI, S BARONTI, M SELVA. Improving component substitution pansharpening through multivariate regression of ms+pan data. IEEE Transactions on Geoscience and Remote Sensing, 45, 3230-3239(2007).

[8] A GARZELLI, F NENCINI, L CAPOBIANCO. Optimal mmse pan sharpening of very high resolution multispectral images. IEEE Transactions on Geoscience and Remote Sensing, 46, 228-236(2008).

[9] Jing ZHANG, Hongtao CHEN, Fan LIU. Remote sensing image fusion based on multivariate empirical mode decomposition and weighted least squares filter. Acta Photonica Sinica, 48, 0510003(2019).

[10] Y KIM, C LEE, D HAN et al. Improved additive-wavelet image fusion. IEEE Geoscience and Remote Sensing Letters, 8, 263-267(2011).

[11] P S PRADHAN, R L KING, N H YOUNAN et al. Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion. IEEE Transactions on Geoscience and Remote Sensing, 44, 3674-3686(2006).

[12] B AIAZZI, L ALPARONE, S BARONTI et al. An mtf-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas, 90-94(2003).

[13] Fan LIU. Remote sensing image fusion based on wavelet kernel filter and sparse representation(2014).

[14] M R VICINANZA, R RESTAINO, G VIVONE et al. A pansharpening method based on the sparse representation of injected details. IEEE Geoscience and Remote Sensing Letters, 12, 180-184(2015).

[15] Xiaopeng PEI. Remote sensing image fusion based on sparse representation(2018).

[16] F PALSSON, J R SVEINSSON, M O ULFARSSON. A new pansharpening algorithm based on total variation. IEEE Geoscience and Remote Sensing Letters, 11, 318-322(2014).

[17] C DONG, C C LOY, K HE et al. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307(2016).

[18] G MASI, D COZZOLINO, L VERDOLIVA et al. Pansharpening by convolutional neural networks. Remote Sensing, 8, 594(2016).

[19] Q YUAN, Y WEI, X MENG et al. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11, 978-989(2018).

[20] J YANG, X FU, Y HU et al. Pannet: a deep network architecture for pan-sharpening, 5449-5457(2017).

[21] Z SHAO, J CAI. Remote sensing image fusion with deep convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11, 1656-1669(2018).

[22] Q LIU, H ZHOU, Q XU et al. Psgan: a generative adversarial network for remote sensing image pan-sharpening. IEEE Transactions on Geoscience and Remote Sensing, 59, 10227-10242(2021).

[23] F OZCELIK, U ALGANCI, E SERTEL et al. Rethinking CNN-based pansharpening: guided colorization of panchromatic images via GANs. IEEE Transactions on Geoscience and Remote Sensing, 59, 3486-3501(2021).

[24] O RONNEBERGER, P FISCHER, T BROX. U-net: convolutional networks for biomedical image segmentation, 234-241(2015).

[25] H SHEN, M JIANG, J LI et al. Spatial-spectral fusion by combining deep learning and variational model. IEEE Transactions on Geoscience and Remote Sensing, 57, 6169-6181(2019).

[26] L DENG, G VIVONE, C JIN et al. Detail injection-based deep convolutional neural networks for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 59, 6995-7010(2021).

[27] A VASWANI, N SHAZEER, N PARMAR et al. Attention is all you need, 5998-6008(2017).

[28] Z LIU, Y LIN, Y CAO et al. Swin transformer: hierarchical vision transformer using shifted windows, 10012-10022(2021).

[29] L WALD, T RANCHIN, M MANGOLINI. Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing, 63, 691-699(1997).

[30] R YUHAS, A GOETZ, J BOARDMAN. Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm, 147-149(1992).

[31] L WALD. Data fusion: definitions and architectures: fusion of images of different spatial resolutions(2002).

[32] J ZHOU, D CIVCO, J SILANDER. A wavelet transform method to merge landsat tm and spot panchromatic data. International Journal of Remote Sensing, 19, 743-757(1998).

[33] L ALPARONE, S BARONTI, A GARZELLI et al. A global quality measurement of pan-sharpened multispectral imagery. IEEE Geoscience and Remote Sensing Letters, 1, 313-317(2004).

[34] A GARZELLI, F NENCINI. Hypercomplex quality assessment of multi/hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 6, 662-665(2009).

[35] B AIAZZI, L ALPARONE, S BARONTI et al. Full-scale assessment of pansharpening methods and data products, 9244, 1-12(2014).

[36] D LEI, H CHEN, L ZHANG et al. Nlrnet: an efficient nonlocal attention resnet for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-13(2021).