Algorithm for Stereo Matching Based on Multi-Task Learning

Yufeng Wang; Hongwei Wang; Yu Liu; Mingquan Yang; Jicheng Quan

doi:10.3788/LOP202158.0415010

Journals >Laser & Optoelectronics Progress >Volume 58 >Issue 4 >Page 0415010 > Article

Laser & Optoelectronics Progress
Vol. 58, Issue 4, 0415010 (2021)

Algorithm for Stereo Matching Based on Multi-Task Learning

Yufeng Wang^1、2, Hongwei Wang^2、3、*, Yu Liu², Mingquan Yang², and Jicheng Quan^1、2、*

Author Affiliations

¹University of Naval Aviation, Yantai, Shandong 264001, China

²Aviation University of Air Force, Changchun, Jilin 130022, China

³Information Engineering University, Zhengzhou, Henan 450001, China

show less

DOI: 10.3788/LOP202158.0415010 Cite this Article Set citation alerts

Yufeng Wang, Hongwei Wang, Yu Liu, Mingquan Yang, Jicheng Quan. Algorithm for Stereo Matching Based on Multi-Task Learning[J]. Laser & Optoelectronics Progress, 2021, 58(4): 0415010 Copy Citation Text

show less

Fig. 1. Overall architecture of proposed algorithm

Download full size

Fig. 2. Architecture comparison of SPP and SPPSA. (a) SPP; (b) SPPSA

Download full size

Fig. 3. Architecture of FEM

Download full size

Fig. 4. Architecture of DRM

Download full size

Fig. 5. Visual results of proposed algorithm. (a) Left image; (b) edge 1; (c) edge 2; (d) feature consistency map; (e) error map; (f) disparity map

Download full size

Number of iterations	E_ep /pixel	E_D1 /%	R₁ /%	R₃ /%	R₅ /%	f_run /(frame·s^-1)
1	1.02	4.12	11.67	4.63	3.38	15.84
2	0.85	3.56	10.06	4.13	2.82	12.22
3	0.83	3.52	9.85	4.02	2.80	8.64
4	0.82	3.46	9.62	3.87	2.81	6.22

Table 1. Performance evaluation of algorithm under each number of DRM iterations

Number of iterations	E_ep /pixel	E_D1 /%	R₁ /%	R₃ /%	R₅ /%	f_run /(frame·s^-1)
ResBlock	0.87	3.75	10.48	4.32	2.93	13.37
SPP	0.86	3.62	10.00	4.22	2.88	12.64
SPPSA	0.85	3.56	10.06	4.13	2.82	12.22

Table 2. Performance evaluation of algorithm under different FEM settings

Number of iterations	E_ep /pixel	E_D1 /%	R₁ /%	R₃ /%	R₅ /%	f_run /(frame·s^-1)
None	1.05	4.48	12.36	4.94	3.52	14.65
Con	1.00	4.17	11.65	4.80	3.32	13.58
edge 1+Con	0.88	3.84	10.71	4.35	2.76	12.46
edge 2+Con	0.89	3.85	10.65	4.37	2.98	13.23
edge 1+edge 2+Con	0.85	3.56	10.06	4.13	2.82	12.22

Table 3. Performance evaluation of algorithm under different auxiliary task settings

Training loss	E_ep /pixel	E_D1 /%	R₁ /%	R₃ /%	R₅ /%
ap	1.65	8.27	27.96	8.65	6.02
ap+edge	1.48	7.51	25.75	7.89	5.36
ap+Con	1.32	6.56	21.72	6.86	4.75
ap+edge+Con	1.28	6.04	19.95	6.23	4.09
SL1+edge+Con	0.71	2.56	13.25	2.87	1.69

Table 4. Performance evaluation under different training losses

Algorithm	E_D1(All) /%			E_D1(Noc) /%			Runtime /s
Algorithm	bg	fg	All area	bg	fg	All area	Runtime /s
M2S_CSPN^[18]	1.51	2.88	1.74	1.40	2.67	1.61	0.50
EdgeStereo-V2^[21]	1.84	3.30	2.08	1.69	2.94	1.89	0.32
WSMCnet^[17]	1.72	4.19	2.13	1.51	3.57	1.85	0.39
SegStereo^[19]	1.88	4.07	2.25	1.76	3.70	2.08	0.60
PSMNet^[12]	1.86	4.62	2.32	1.71	4.31	2.14	0.41
iResNet-i2^[9]	2.25	3.40	2.44	4.11	3.72	4.05	0.12
CRL^[8]	2.48	3.59	2.67	2.32	3.12	2.45	0.47
GC-net^[11]	2.21	6.16	2.87	2.02	5.58	2.61	0.90
MBFnet^[22]	2.59	4.80	2.96	2.22	4.14	2.54	0.05
SGM-Net^[6]	2.66	8.64	3.66	2.23	7.44	3.09	67.00
MC-CNN-arct^[2]	2.89	8.88	3.89	2.48	7.64	3.33	67.00
DispNetC^[7]	4.32	4.41	4.34	4.11	3.72	4.05	0.06
Proposed algorithm	2.07	4.01	2.39	1.89	3.69	2.19	0.09

Table 5. Performance evaluation of each algorithm on KITTI2015 test dataset

Dataset	K-val		MQ-val		ETH-val
Dataset	E_ep /pixel	E_D1 /%	E_ep /pixel	E_D1 /%	E_ep /pixel	ED1 /%
Pretrained	1.48	7.05	0.72	4.99	0.65	4.77
K-train	0.71	2.56	1.06	7.20	0.51	1.66
MQ-train	1.72	9.38	0.47	2.58	0.59	2.69
ETH-train	1.84	10.85	1.32	8.36	0.32	1.26
KME-train	0.78	3.01	0.49	2.61	0.36	1.33

Table 6. Performance evaluation on each dataset

Yufeng Wang, Hongwei Wang, Yu Liu, Mingquan Yang, Jicheng Quan. Algorithm for Stereo Matching Based on Multi-Task Learning[J]. Laser & Optoelectronics Progress, 2021, 58(4): 0415010

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information