Real-Time Stereo Matching Algorithm with Hierarchical Refinement

Yufeng Wang; Hongwei Wang; Yu Liu; Mingquan Yang; Jicheng Quan

doi:10.3788/AOS202040.0915002

[1] Scharstein D, Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 47, 7-42(2002).

[2] Heise P, Jensen B, Klose S et al. Fast dense stereo correspondences by binary locality sensitive hashing. [C]∥The IEEE International Conference on Robotics and Automation (ICRA), May 26-30, 2015, Seattle, WA, USA. New York: IEEE, 105-110(2015).

[3] Taniai T, Matsushita Y, Sato Y et al. Continuous 3D label stereo matching using local expansion moves[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 2725-2739(2018).

[4] Žbontar J. LeCun Y. Computing the stereo matching cost with a convolutional neural network. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE, 1592-1599(2015).

[5] Žbontar J. LeCun Y. Stereo matching by training a convolutional neural network to compare image patches[J]. The Journal of Machine Learning Research, 17, 2287-2318(2016).

[6] Park H, Lee K M. Look wider to match image patches with convolutional neural networks[J]. IEEE Signal Processing Letters, 24, 1788-1792(2017).

[7] Chen Z Y, Sun X, Wang L et al. A deep visual correspondence embedding model for stereo matching costs. [C]∥The IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE, 972-980(2015).

[8] Luo W J, Schwing A G, Urtasun R. Efficient deep learning for stereo matching. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 5695-5703(2016).

[9] Xiao J S, Tian H, Zou W T et al. Stereo matching based on convolutional neural network[J]. Acta Optica Sinica, 38, 0815017(2018).

[10] Seki A, Pollefeys M. Patch based confidence prediction for dense disparity map[C]∥Proceedings of the British Machine Vision Conference(BMVC), September 19-22, 2016, York, UK., 23, 13.

[11] Seki A, Pollefeys M. SGM-nets: semi-global matching with neural networks. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI. New York: IEEE, 6640-6649(2017).

[12] Guney F, Geiger A. Displets: resolving stereo ambiguities using object knowledge. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE, 4165-4175(2015).

[13] Mayer N, Ilg E, Hausser P et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 4040-4048(2016).

[14] Pang J H, Sun W X, Ren J S et al. Cascade residual learning: a two-stage convolutional neural network for stereo matching. [C]∥The IEEE International Conference on Computer Vision Workshops (ICCVW), October 22-29, 2017, Venice, Italy. New York: IEEE, 878-886(2017).

[15] Liang Z F, Feng Y L, Guo Y L et al. Learning for disparity estimation through feature constancy. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 18-23, 2018, Salt Lake City, UT. New York: IEEE, 2811-2820(2018).

[16] Jie Z Q, Wang P F, Ling Y G et al. Left-right comparative recurrent model for stereo matching. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), June 18-23, 2018, Salt Lake City, UT. New York: IEEE, 3838-3846(2018).

[17] Kendall A, Martirosyan H, Dasgupta S et al. End-to-end learning of geometry and context for deep stereo regression. [C]∥The IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice. New York: IEEE, 66-75(2017).

[18] Chang J R, Chen Y S. Pyramid stereo matching network. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), June 18-23, 2018, Salt Lake City, UT. New York: IEEE, 5410-5418(2018).

[19] Nie G Y, Cheng M M, Liu Y et al. Multi-level context ultra-aggregation for stereo matching. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 16-20, 2019, Long Beach, California. New York: IEEE, 3283-3291(2019).

[20] Du X Z, El-Khamy M. -04-19)[2019-10-31]. http: ∥cn.arxiv., org/abs/1904, 09099(2019).

[21] Tulyakov S, Ivanov A, Fleuret F. Practical deep stereo (PDS): toward applications-friendly deep stereo matching[M]. ∥Bengio S, Wallach H, Larochelle H, et al. Advances in neural information processing systems 31. New York: Curran Associates, 5871-5881(2018).

[22] Duggal S, Wang S L, Ma W C et al. DeepPruner: learning efficient stereo matching via differentiable PatchMatch. [C]∥The IEEE International Conference on Computer Vision(ICCV), October 20-26, 2019, Seoul, Korea. New York: IEEE, 4384-4393(2019).

[23] Wang Y F, Wang H W, Yu G et al. Stereo matching algorithm based on three-dimensional convolutional neural network[J]. Acta Optica Sinica, 39, 1115001(2019).

[24] Guo X Y, Yang K, Yang W K et al. Group-wise correlation stereo network. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 16-20, 2019, Long Beach, California. New York: IEEE, 3273-3282(2019).

[25] Cheng X J, Wang P. -10-04)[2019-10-31]. http: ∥cn.arxiv., org/abs/1810, 02695(2018).

[26] Zhang F H, Prisacariu V, Yang R G et al. GA-Net: guided aggregation net for end-to-end stereo matching. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 16-20, 2019, Long Beach, California. New York: IEEE, 185-194(2019).

[27] Khamis S, Fanello S, Rhemann C et al. StereoNet: guided hierarchical refinement for real-time edge-aware depth prediction. [C]∥The European Conference on Computer Vision (ECCV), Sepember 8-14, 2018, Munich, Germany. Cham: Springer, 573-590(2018).

[28] Tonioni A, Tosi F, Poggi M et al. Real-time self-adaptive deep stereo. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 16-20, 2019, Long Beach, California. New York: IEEE, 195-204(2019).

[29] Zhao H S, Shi J P, Qi X J et al. Pyramid scene parsing network. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI. New York: IEEE, 6230-6239(2017).

[30] Ioffe S. -03-02)[2019-10-31]. https: ∥arxiv.xilesou., top/abs/1502, 03167(2015).

[31] Maas A L, Hannun A Y, Ng A Y. Rectifier nonlinearities improve neural network acoustic models. [C]∥The 30th International Conference on Machine Learning (ICML13), June 16-21, 2013, Atlanta, GA, USA. Cambridge: JMLR, 28, 1498-1503(2013).

[32] He K M, Zhang X Y, Ren S Q et al. Deep residual learning for image recognition. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 770-778(2016).

[33] Girshick R. Fast R-CNN. [C]∥The IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE, 1440-1448(2015).

[34] Menze M, Geiger A. Object scene flow for autonomous vehicles. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE, 3061-3070(2015).

[35] Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite. [C]∥The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), June 16-21, 2012, Providence, RI. New York: IEEE, 3354-3361(2012).

[36] Kingma D P. -01-30)[2019-10-31]. http: ∥cn.arxiv., org/abs/1412, 6980(2017).