Depth map super-resolution with cascaded pyramid structure

Fu Xuwen; Zhang Xudong; Zhang Jun; Sun Rui

doi:10.12086/oee.2019.180587

[1] Ruiz-Sarmiento J R, Galindo C, Gonzalez J. Improving human face detection through TOF cameras for ambient intelligence applications[C]//Proceedings of the 2nd International Sympo-sium on Ambient Intelligence, 2011: 125–132.

[2] Hui T W, Loy C C, Tang X O. Depth map super-resolution by deep multi-scale guidance[C]//Proceedings of the 14th Euro-pean Conference on Computer Vision, 2016: 353–369.

[3] Ferstl D, Reinbacher C, Ranftl R, et al. Image guided depth upsampling using anisotropic total generalized varia-tion[C]//Proceedings of 2013 IEEE International Conference on

Computer Vision, 2013: 993–1000.

[4] He K M, Sun J, Tang X O. Guided image filter-ing[C]//Proceedings of the 11th European Conference on Computer Vision, 2010: 1–14.

[7] Richardt C, Stoll C, Dodgson N A, et al. Coherent spatiotem-poral filtering, upsampling and rendering of RGBZ videos[J]. Computer Graphics Forum, 2012, 31(2): 247–256.

[8] Shen X Y, Zhou C, Xu L, et al. Mutual-structure for joint filter-ing[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 3406–3414.

[9] Kiechle M, Hawe S, Kleinsteuber M. A joint intensity and depth co-sparse analysis model for depth map su-per-resolution[C]//Proceedings of 2013 IEEE International Conference on Computer Vision, 2013: 1545–1552.

[10] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems, 2012: 1097–1105.

[11] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal net-works[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015: 91–99.

[12] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition, 2015: 3431–3440.

[13] Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307.

[14] Wang Z W, Liu D, Yang J C, et al. Deep networks for image super-resolution with sparse prior[C]//Proceedings of IEEE In-ternational Conference on Computer Vision, 2015: 370–378.

[15] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1646–1654.

[16] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Confe-rence on Computer Vision and Pattern Recognition, 2017: 2261–2269.

[17] Shi W Z, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolu-tional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874–1883.

[18] Lai W S, Huang J B, Ahuja N, et al. Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Rec-ognition, 2017: 5835–5843.

[19] Kopf J, Cohen M F, Lischinski D, et al. Joint bilateral upsam-pling[J]. ACM Transactions on Graphics, 2007, 26(3): 96.

[20] Yang Q, Yang R, Davis J, et al. Spatial-depth super resolution for range images[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2007: 1–8.

[21] Chan D, Buisman H, Theobalt C, et al. A noise-aware filter for real-time depth upsampling[C]//Proceedings of Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications, 2008.

[22] Lu J J, Forsyth D. Sparse depth super resolu-tion[C]//Proceedings of 2005 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 2245–2253.

[23] Yuan L, Jin X, Li Y G, et al. Depth map super-resolution via low-resolution depth guided joint trilateral up-sampling[J]. Journal of Visual Communication and Image Representation, 2017, 46: 280–291.

[24] Diebel J, Thrun S. An application of Markov random fields to range sensing[C]//Advances in Neural Information Processing Systems, 2005: 291–298.

[25] Park J, Kim H, Tai Y W, et al. High quality depth map upsam-pling for 3D-TOF cameras[C]//Proceedings of 2011 IEEE Inter-national Conference on Computer Vision, 2011: 1623–1630.

[26] Aodha O M, Campbell N D F, Nair A, et al. Patch based synthe-sis for single depth image super-resolution[C]//Proceedings of the 12th European Conference on Computer Vision, 2012: 71–84.

[27] Yang J Y, Ye X C, Li K, et al. Color-guided depth recovery from RGB-D data using an adaptive autoregressive model[J]. IEEE Transactions on Image Processing, 2014, 23(8): 3443–3458.

[28] Lei JJ, Li L L, Yue H J, et al. Depth map super-resolution con-sidering view synthesis quality[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1732–1745.

[29] Denton E, Chintala S, Szlam A, et al. Deep generative image models using a Laplacian pyramid of adversarial net-works[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015: 1486–1494.

[30] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770–778.

[31] He K M, Zhang X Y, Ren S Q, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classifica-tion[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 1026–1034.

[32] Scharstein D, Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47(1–3): 7–42.

[33] Scharstein D, Pal C. Learning conditional random fields for stereo[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007: 1–8.

[34] Scharstein D, Hirschmüller H, Kitajima Y, et al. High-resolution stereo datasets with subpixel-accurate ground truth[C]//Proceedings of the 36th German Conference on Pat-tern Recognition, 2014: 31–42.

[35] Handa A, Whelan T, McDonald J, et al. A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM[C]//Proceedings of 2014 IEEE International Conference on Robotics and Auto-mation, 2014: 1524–1531.

[36] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010: 249–256.

[37] Kingma D, Ba J. Adam: a method for stochastic optimiza-tion[C]//Proceedings of the 3rd International Conference on Learning Representations, 2014.

[38] Ferstl D, Rüther M, Bischof H. Variational depth superresolution using example-based edge representations[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 513–521.

[39] Xie J, Feris R S, Sun M T. Edge guided single depth image super resolution[C]//Proceedings of 2014 IEEE International Conference on Image Processing, 2014: 3773–3777.

[40] Song X B, Dai Y C, Qin X Y. Deep depth super-resolution: learning depth super-resolution using deep convolutional neural network[C]//Proceedings of the 13th Asian Conference on Computer Vision, 2017: 360–376.