SAR-Assisted Optical Remote Sensing Image Cloud Removal Method Based on Deep Learning

Mengyao Wang; Xiangchao Meng; Feng Shao; Randi Fu

doi:10.3788/AOS202141.1228002

[1] Shen H F, Li X H, Cheng Q et al. Missing information reconstruction of remote sensing data: a technical review[J]. IEEE Geoscience and Remote Sensing Magazine, 3, 61-85(2015).

[2] Wang L L, Qu J J, Xiong X X et al. A new method for retrieving band 6 of Aqua MODIS[J]. IEEE Geoscience and Remote Sensing Letters, 3, 267-270(2006). http://ieeexplore.ieee.org/document/1621093

[3] Rakwatin P, Takeuchi W, Yasuoka Y. Restoration of Aqua MODIS band 6 using histogram matching and local least squares fitting[J]. IEEE Transactions on Geoscience and Remote Sensing, 47, 613-627(2009).

[4] Shen H F, Zeng C, Zhang L P. Recovering reflectance of AQUA MODIS band 6 based on within-class local fitting[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 4, 185-192(2011).

[5] Gladkova I, Grossberg M D, Shahriar F et al. Quantitative restoration for MODIS band 6 on Aqua[J]. IEEE Transactions on Geoscience and Remote Sensing, 50, 2409-2416(2012). http://ieeexplore.ieee.org/document/6095345

[6] Shen H F, Li X H, Zhang L P et al. Compressed sensing-based inpainting of Aqua moderate resolution imaging spectroradiometer band 6 using adaptive spectrum-weighted sparse Bayesian dictionary learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 52, 894-906(2014). http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6484934&queryText%3DCompressed+Sensing-Based+Inpainting+of+Aqua+Moderate+Resolution+Imaging+Spectroradiometer+Band+6+Usi...

[7] Zhang C, Li W, Travis D. Gaps-fill of SLC-off Landsat ETM + satellite image using a geostatistical approach[J]. International Journal of Remote Sensing, 28, 5103-5122(2007). http://www.tandfonline.com/doi/full/10.1080/01431160701250416

[8] Yu C, Chen L F, Su L et al. Kriging interpolation method and its application in retrieval of MODIS aerosol optical depth[C]∥2011 19th International Conference on Geoinformatics, June 24-26, 2011, Shanghai, China., 1-6(2011).

[9] Maalouf A, Carre P, Augereau B et al. A bandelet-based inpainting technique for clouds removal from remotely sensed images[J]. IEEE Transactions on Geoscience and Remote Sensing, 47, 2363-2371(2009). http://ieeexplore.ieee.org/document/4804804

[10] Mendez-Rial R, Calvino-Cancela M, Martin-Herrero J. Anisotropic inpainting of the hypercube[J]. IEEE Geoscience and Remote Sensing Letters, 9, 214-218(2012).

[11] Shen H F, Zhang L P. A MAP-based algorithm for destriping and inpainting of remotely sensed images[J]. IEEE Transactions on Geoscience and Remote Sensing, 47, 1492-1502(2009).

[12] Cheng Q, Shen H F, Zhang L P et al. Inpainting for remotely sensed images with a multichannel nonlocal total variation model[J]. IEEE Transactions on Geoscience and Remote Sensing, 52, 175-187(2014). http://smartsearch.nstl.gov.cn/paper_detail.html?id=62805c79cef7ee79cb82ad52715e3c5d

[13] Lin C H, Tsai P H, Lai K H et al. Cloud removal from multitemporal satellite images using information cloning[J]. IEEE Transactions on Geoscience and Remote Sensing, 51, 232-241(2013). http://ieeexplore.ieee.org/document/6213540

[14] Zeng C, Shen H F, Zhang L P. Recovering missing pixels for Landsat ETM + SLC-off imagery using multi-temporal regression analysis and a regularization method[J]. Remote Sensing of Environment, 131, 182-194(2013). http://www.sciencedirect.com/science/article/pii/S0034425712004786

[15] Shen H F, Wu J G, Cheng Q et al. A spatiotemporal fusion based cloud removal method for remote sensing images with land cover changes[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12, 862-874(2019).

[16] Li Z W, Shen H F, Cheng Q et al. Thick cloud removal in high-resolution satellite images using stepwise radiometric adjustment and residual correction[J]. Remote Sensing, 11, 1925-1943(2019). http://www.researchgate.net/publication/335238436_Thick_Cloud_Removal_in_High-Resolution_Satellite_Images_Using_Stepwise_Radiometric_Adjustment_and_Residual_Correction

[17] Qin Y, Fu Z L, Li Y. Algorithm for cloud restoring based on texture synthesis algorithm[J]. Journal of Nanchang University (Engineering & Technology), 39, 194-199(2017).

[18] Zhao S. The image restoration algorithm based on texture synthesis[D]. Chengdu: University of Electronic Science and Technology of China(2014).

[19] Chen J, Jönsson P, Tamura M et al. A simple method for reconstructing a high-quality NDVI time-series data set based on the Savitzky-Golay filter[J]. Remote Sensing of Environment, 91, 332-344(2004).

[20] Lorenzi L, Melgani F, Mercier G. Missing-area reconstruction in multispectral images under a compressive sensing perspective[J]. IEEE Transactions on Geoscience and Remote Sensing, 51, 3998-4008(2013).

[21] Li X H, Shen H F, Zhang L P et al. Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 52, 7086-7098(2014).

[22] Hoan N T, Tateishi R. Cloud removal of optical image using SAR data for ALOS application: experimenting on simulated ALOS data[J]. Journal of the Remote Sensing Society of Japan, 29, 410-417(2009). http://d.wanfangdata.com.cn/Conference/WFHYXW328035

[23] Huang B, Li Y, Han X Y et al. Cloud removal from optical satellite imagery with SAR imagery using sparse representation[J]. IEEE Geoscience and Remote Sensing Letters, 12, 1046-1050(2015). http://smartsearch.nstl.gov.cn/paper_detail.html?id=4e5799e6d8fa6175b33dcd0533b8f47e

[24] Liu L, Lei B. Can SAR images and optical images transfer with each other?[C]∥2018 IEEE International Geoscience and Remote Sensing Symposium, July 22-27, 2018, Valencia, Spain., 7019-7022(2018).

[25] Reyes M F, Auer S, Merkle N et al. SAR-to-optical image translation based on conditional generative adversarial networks: optimization, opportunities and limits[J]. Remote Sensing, 11, 2067-2085(2019). http://www.researchgate.net/publication/335631346_SAR-to-Optical_Image_Translation_Based_on_Conditional_Generative_Adversarial_Networks-Optimization_Opportunities_and_Limits

[26] Grohnfeldt C, Schmitt M, Zhu X X. A conditional generative adversarial network to fuse sar and multispectral optical data for cloud removal from Sentinel-2 images[C]∥2018 IEEE International Geoscience and Remote Sensing Symposium, July 22-27, 2018, Valencia, Spain., 1726-1729(2018).

[27] Goodfellow I, Pouget-Abadie J, Mirza M et al. Generative adversarial nets. [C]∥Neural Information Processing Systems, December 8-13, 2014, Montreal, Canada. New York: Curran Associates, 2672-2680(2014).

[28] Wang E D, Qi K, Li X P et al. Semantic segmentation of remote sensing image based on neural network[J]. Acta Optica Sinica, 39, 1210001(2019).

[29] Xiao J S, Liu E Y, Zhu L et al. Improved image super-resolution algorithm based on convolutional neural network[J]. Acta Optica Sinica, 37, 0318011(2017).

[30] Li Y, Fu R D, Meng X C et al. A SAR-to-optical image translation method based on conditional generation adversarial network (cGAN)[J]. IEEE Access, 8, 60338-60343(2020). http://ieeexplore.ieee.org/document/9017930/

[31] Isola P, Zhu J Y, Zhou T H et al. Image-to-image translation with conditional adversarial networks[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA., 5967-5976(2017).

[32] Zuhlke M, Fomferra N, Brockmann C et al. SNAP (Sentinel Application Platform) and the ESA Sentinel 3 Toolbox. [C]∥Sentinel-3 for Science Workshop, June 2-5, 2015, Venice, Italy. Oxford: ESA Special Publication(SP-734), 978, 21(2015).

[33] Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks[J]. Science, 313, 504-507(2006). http://logcom.oxfordjournals.org/cgi/ijlink?linkType=ABST&journalCode=sci&resid=313/5786/504

[34] Zeiler M D, Taylor G W, Fergus R. Adaptive deconvolutional networks for mid and high level feature learning[C]∥2011 International Conference on Computer Vision, November 6-13, 2011, Barcelona, Spain., 2018-2025(2011).

[35] Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. [C]∥Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, October 5-9, 2015, Munich, Germany. Lecture Notes in Electrical Engineering, Singapore: Springer, 9351, 234-241(2015).

[36] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. [C]∥Proceedings of the 32nd International Conference on Machine Learning(ICML), July 6-11, 2015, Lille, France. JMLR Workshop and Conference Proceedings, 37, 448-456(2015).

[37] Wang Z, Bovik A C, Sheikh H R et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 13, 600-612(2004).

[38] Kingma D P. -12-22)[2020-10-18]. https:∥arxiv., org/abs/1412, 6980(2014).