• Opto-Electronic Engineering
  • Vol. 51, Issue 6, 240093-1 (2024)
Yulong Li1, Yeyao Chen1, Yueli Cui2, and Mei Yu1,*
Author Affiliations
  • 1Faculty of Information Science and Engineering, Ningbo University, Ningbo, Zhejiang 315211, China
  • 2School of Electronic and Information Engineering, Taizhou University, Taizhou, Zhejiang 318000, China
  • show less
    DOI: 10.12086/oee.2024.240093 Cite this Article
    Yulong Li, Yeyao Chen, Yueli Cui, Mei Yu. LF-UMTI: unsupervised multi-exposure light field image fusion based on multi-scale spatial-angular interaction[J]. Opto-Electronic Engineering, 2024, 51(6): 240093-1 Copy Citation Text show less
    Overall network diagram of the proposed method
    Fig. 1. Overall network diagram of the proposed method
    Light field spatial angular feature extraction module
    Fig. 2. Light field spatial angular feature extraction module
    Light field information interaction module
    Fig. 3. Light field information interaction module
    (a) Feature fusion module; (b) Bottleneck residual module
    Fig. 4. (a) Feature fusion module; (b) Bottleneck residual module
    Light field reconstruction module
    Fig. 5. Light field reconstruction module
    Some scene examples in the benchmark dataset established in this work
    Fig. 6. Some scene examples in the benchmark dataset established in this work
    Subjective comparison results of different methods on the established benchmark dataset
    Fig. 7. Subjective comparison results of different methods on the established benchmark dataset
    Subjective comparison of depth maps estimated from the fused light field images obtained with different fusion methods
    Fig. 8. Subjective comparison of depth maps estimated from the fused light field images obtained with different fusion methods
    Subjective comparison results of the main network structure ablation experiments
    Fig. 9. Subjective comparison results of the main network structure ablation experiments
    Subjective comparison results of angle consistency loss ablation experiments
    Fig. 10. Subjective comparison results of angle consistency loss ablation experiments
    MethodSD↑MEFSSIM↑QcvSF↑QabfQniceNMI↑AG↑Rank↓
    DISFT_EF[8] 58.2897(5) 0.9610(3) 679.7548(8) 23.5287(2) 0.7489(3) 0.8118(10) 0.6411(8) 5.4570(7) 46
    GFF[7] 60.2416(3)0.9624(2) 592.7575(10)25.8524(1)0.7648(1) 0.8154(7) 0.6074(9) 5.7301(3) 36
    MEFAW[9] 54.2736(6)0.9639(1) 625.0993(9) 22.2241(3) 0.7507(2) 0.8163(6) 0.5215(10) 5.5581(4) 41
    PAS_MEF[10] 48.7852(10) 0.8887(9) 517.8809(7) 16.2884(7) 0.5452(6) 0.8146(8) 0.6610(7) 5.5482(5) 59
    DeepFuse[20] 53.3335(7) 0.9040(6) 321.1736(4) 16.3365(6) 0.5408(7) 0.8202(3) 0.8482(3) 5.4888(6) 42
    PMGI[27] 51.0411(9) 0.8933(8) 361.4652(6) 16.2283(8) 0.5209(8) 0.8194(4) 0.8285(4) 5.4126(8) 55
    U2Fusion[26] 52.7949(8) 0.8973(7) 316.6257(3) 15.5335(10) 0.5061(9) 0.8183(5) 0.7999(5) 5.3375(10) 57
    TransMEF[21]60.6602(2) 0.9328(5)281.9047(2) 18.1766(5) 0.6264(5)0.8210(1)0.8758(1) 6.0909(2)23
    FFMEF[22] 58.9095(4) 0.8595(10) 334.6085(5) 16.0121(9) 0.4360(10) 0.8135(9) 0.7276(6) 5.4097(9) 62
    Proposed67.5961(1) 0.9491(4)238.4925(1) 20.5533(4) 0.6870(4)0.8205(2)0.8510(2)6.7186(1)19
    Table 1. Comparison of objective indicators among different methods on 50 testing scenarios
    Method SD↑MEFSSIM↑QcvSF↑QabfQniceNMI↑AG↑
    w/o C-ASA74.6707 0.8747 313.7698 16.8144 0.4682 0.8143 0.6906 5.7951
    w/o T-INN 40.1623 0.7105 948.2726 16.4418 0.3252 0.8095 0.4769 6.1205
    w/o MK 66.819 0.8999 252.7716 19.3284 0.5027 0.8142 0.6666 6.3629
    Proposed 67.59610.9491238.492520.55330.6870.82050.8516.7186
    Table 2. Comparative results of objective indicators of the main network structure ablation experiments in this article
    Method GFF[7] MEFAW[9] DISFT_EF[8] DeepFuse[20] PMGI[27] TransMEF[21] Proposed
    Runtime/s 13.6808 30.5074 23.0349 14.8176 6.9727 11.2749 16.8701
    Params/M - - - 0.283 0.0401 18.1696 0.9734
    Platform MATLAB MATLAB MATLAB Pytorch Pytorch Pytorch Pytorch
    Device CPU CPU CPU GPU GPU GPU GPU
    Table 3. Comparison results of running time and parameter quantities of different methods
    Yulong Li, Yeyao Chen, Yueli Cui, Mei Yu. LF-UMTI: unsupervised multi-exposure light field image fusion based on multi-scale spatial-angular interaction[J]. Opto-Electronic Engineering, 2024, 51(6): 240093-1
    Download Citation