Color image multi-scale guided depth image super-resolution reconstruction

Yu Shuxia; Hu Liangmei; Zhang Xudong; Fu Xuwen

doi:10.12086/oee.2020.190260

[1] Palacios J M, Sagüés C, Montijano E, et al. Human-computer interaction based on hand gestures using RGB-D sensors[J]. Sensors, 2013, 13(9): 11842–11860.

[2] Nguyen T N, Huynh H H, Meunier J. 3D reconstruction with time-of-flight depth camera and multiple mirrors[J]. IEEE Access, 2018, 6: 38106–38114.

[3] Yamamoto S. Development of inspection robot for nuclear power plant[C]//Proceedings of 1992 IEEE International Conference on Robotics and Automation, Nice, France, 1992: 1559?1566.

[4] Kolb A, Barth E, Koch R, et al. Time-of-flight cameras in computer graphics[J]. Computer Graphics Forum, 2010, 29(1): 141–159.

[5] Xie J, Feris R S, Yu S S, et al. Joint super resolution and denoising from a single depth image[J]. IEEE Transactions on Multimedia, 2015, 17(9): 1525–1537.

[6] Mandal S, Bhavsar A, Sao A K. Noise adaptive super-resolution from single image via non-local mean and sparse representation[J]. Signal Processing, 2017, 132: 134–149.

[7] Aodha O M, Campbell N D F, Nair A, et al. Patch based synthesis for single depth image super-resolution[C]//Proceedings of the 12th European Conference on Computer Vision, Florence, Italy, 2012: 71–84.

[8] Li J, Lu Z C, Zeng G, et al. Similarity-aware patchwork assembly for depth image super-resolution[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014: 3374–3381.

[9] Xie J, Feris R S, Sun M T. Edge-guided single depth image super resolution[J]. IEEE Transactions on Image Processing, 2016, 25(1): 428–438.

[10] Chen B L, Jung C. Single depth image super-resolution using convolutional neural networks[C]//Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, AB, Canada, 2018: 1473–1477.

[11] Liu W, Chen X G, Yang J, et al. Robust color guided depth map restoration[J]. IEEE Transactions on Image Processing, 2017, 26(1): 315–327.

[12] Kiechle M, Hawe S, Kleinsteuber M. A joint intensity and depth co-sparse analysis model for depth map super-resolution[C]//Proceedings of 2013 International Conference on Computer Vision, Sydney, NSW, Australia, 2013: 1545–1552.

[13] Li Y, Min D B, Do M N, et al. Fast guided global interpolation for depth and motion[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 717–733.

[14] Park J, Kim H, Tai Y W, et al. High quality depth map upsampling for 3D-tof cameras[C]//Proceedings of 2011 IEEE International Conference on Computer Vision, Barcelona, Spain, 2011: 1623–1630.

[15] Li W, Zhang X D. Depth image super-resolution reconstruction based on convolution neural network[J]. Journal of Electronic Measurement and Instrumentation, 2017, 31(12): 1918–1928.

[16] Xiao Y, Cao X, Zhu X Y, et al. Joint convolutional neural pyramid for depth map super-resolution[Z]. arXiv:1801.00968, 2018.

[17] Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278–2324.

[18] Scharstein D, Szeliski R, Zabih R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[C]//Proceedings of 2001 IEEE Workshop on Stereo and Multi-Baseline Vision, Kauai, HI, USA, 2001: 131–140.

[19] Richardt C, Stoll C, Dodgson N A, et al. Coherent spatiotemporal filtering, upsampling and rendering of rgbz videos[J]. Computer Graphics Forum, 2012, 31(2): 247–256.

[20] Lu S, Ren X F, Liu F. Depth enhancement via low-rank matrix completion[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014: 3390–3397.

[21] Handa A, Whelan T, McDonald J, et al. A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM[C]//Proceedings of 2014 IEEE International Conference on Robotics and Automation, Hong Kong, China, 2014: 1524–1531.

[22] He K M, Zhang X Y, Ren S Q, et al. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 1026–1034.

[23] Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307.

[24] Hui T W, Loy C C, Tang X O. Depth map super-resolution by deep multi-scale guidance[C]//Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 353–369.

[25] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016: 1646–1654.

[26] Lai W S, Huang J B, Ahuja N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017: 624–632.