Colorful 3D Reconstruction from Single Image Based on Deep Learning

Yuzheng Zhu; Yaping Zhang; Qiaosheng Feng

doi:10.3788/LOP202158.1410010

Journals >Laser & Optoelectronics Progress >Volume 58 >Issue 14 >Page 1410010 > Article

Laser & Optoelectronics Progress
Vol. 58, Issue 14, 1410010 (2021)

Colorful 3D Reconstruction from Single Image Based on Deep Learning

Yuzheng Zhu, Yaping Zhang^*, and Qiaosheng Feng

Author Affiliations

School of Information Science and Technology, Yunnan Normal University, Kunming, Yunnan 650500, China

show less

DOI: 10.3788/LOP202158.1410010 Cite this Article Set citation alerts

Yuzheng Zhu, Yaping Zhang, Qiaosheng Feng. Colorful 3D Reconstruction from Single Image Based on Deep Learning[J]. Laser & Optoelectronics Progress, 2021, 58(14): 1410010 Copy Citation Text

show less

Fig. 1. Model structure

Download full size

Fig. 2. Reconstruction network of 3D shape

Download full size

Fig. 3. Encoder structure added to CBAM module

Download full size

Fig. 4. Color reconstruction network

Download full size

Fig. 5. SoftRas rendering structure

Download full size

Fig. 6. Comparison of predictions of different 3D models. (a) Original images; (b) 3D-R2N2; (c) PSGN; (d) OccNet; (e) SoftRas_O; (f) designed model

Download full size

Fig. 7. Reconstruction results of different color 3D models. (a1) (a2) Original images; (b1) (b2) results of reconstruction

Download full size

Attention mechanism	IOU	SSIM	MSE
SoftRas_O	0.576	0.548	0.175
CBAM_S	0.589	0.579	0.161
SENet	0.593	0.563	0.186
CBAM	0.593	0.578	0.163

Table 1. Comparison of indicators of different attention mechanism

Category	IOU
Category	SoftRas_O	CBAM 1-2	CBAM 3-4	CBAM 1-4
Airplane	0.603	0.646	0.648	0.648
Bench	0.444	0.461	0.456	0.454
Cabinet	0.616	0.629	0.613	0.633
Car	0.672	0.665	0.669	0.675
Chair	0.481	0.492	0.478	0.493
Display	0.556	0.571	0.554	0.563
Lamp	0.444	0.463	0.460	0.459
Loudspeaker	0.615	0.631	0.635	0.636
Rifle	0.643	0.677	0.683	0.674
Sofa	0.619	0.618	0.628	0.630
Table	0.453	0.487	0.476	0.470
Telephone	0.741	0.744	0.776	0.766
Vessel	0.589	0.611	0.610	0.612
Mean	0.575508	0.59192	0.59123	0.59330

Table 2. Comparison of IOU indicators of different CBAMs and SoftRas_O

Category	IOU
Category	3D-R2N2	OccNet	SoftRas_O	Our model
Airplane	0.426	0.547	0.603	0.648
Bench	0.373	0.452	0.444	0.454
Cabinet	0.676	0.732	0.616	0.633
Car	0.661	0.731	0.672	0.675
Chair	0.439	0.502	0.481	0.493
Display	0.440	0.479	0.556	0.563
Lamp	0.281	0.370	0.444	0.459
Loudspeaker	0.611	0.653	0.615	0.636
Rifle	0.375	0.458	0.643	0.674
Sofa	0.626	0.671	0.619	0.630
Table	0.420	0.506	0.453	0.470
Telephone	0.611	0.709	0.741	0.766
Vessel	0.482	0.521	0.589	0.612
Mean	0.493	0.564	0.571	0.593

Table 3. Comparison of IOU indicators of different models

Category	SSIM		MSE
Category	SoftRas_O	Our model	SoftRas_O	Our model
Airplane	0.627	0.610	0.084	0.090
Bench	0.475	0.493	0.186	0.183
Cabinet	0.565	0.587	0.188	0.182
Car	0.584	0.592	0.197	0.184
Chair	0.442	0.486	0.240	0.222
Display	0.453	0.488	0.264	0.250
Lamp	0.537	0.749	0.156	0.068
Loudspeaker	0.569	0.591	0.218	0.219
Rifle	0.592	0.592	0.081	0.070
Sofa	0.489	0.494	0.230	0.228
Table	0.537	0.548	0.198	0.197
Telephone	0.654	0.673	0.136	0.133
Vessel	0.600	0.617	0.096	0.098
Mean	0.548	0.578	0.175	0.163

Table 4. Comparison results of SSIM and MSE between designed model and SoftRas_O

Yuzheng Zhu, Yaping Zhang, Qiaosheng Feng. Colorful 3D Reconstruction from Single Image Based on Deep Learning[J]. Laser & Optoelectronics Progress, 2021, 58(14): 1410010

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information