Medical Image Fusion Based on Multi-Scale Feature Learning and Edge Enhancement

Wanxin Xiao; Huafeng Li; Yafei Zhang; Minghong Xie; Fan Li

doi:10.3788/LOP202259.0617029

Journals >Laser & Optoelectronics Progress >Volume 59 >Issue 6 >Page 0617029 > Article

Laser & Optoelectronics Progress
Vol. 59, Issue 6, 0617029 (2022)

Medical Image Fusion Based on Multi-Scale Feature Learning and Edge Enhancement

Wanxin Xiao^1、2, Huafeng Li^1、2, Yafei Zhang^1、2、*, Minghong Xie¹, and Fan Li^1、2

Author Affiliations

¹Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming , Yunnan 650500, China

²Yunnan Key Laboratory of Artificial Intelligence, Kunming , Yunnan 650500, China

show less

DOI: 10.3788/LOP202259.0617029 Cite this Article Set citation alerts

Wanxin Xiao, Huafeng Li, Yafei Zhang, Minghong Xie, Fan Li. Medical Image Fusion Based on Multi-Scale Feature Learning and Edge Enhancement[J]. Laser & Optoelectronics Progress, 2022, 59(6): 0617029 Copy Citation Text

show less

Fig. 1. Basic framework of multi-scale feature learning and edge enhancement fusion network

Download full size

Fig. 2. Fusion results obtained by the CNN and NSCT-PAPCNN methods. (a) Source CT image; (b) source MR-T2 image; (c) CNN; (d) NSCT-PRPCNN

Download full size

Fig. 3. Source image samples in testing dataset

Download full size

Fig. 4. Visual comparison of the different methods. (a) Source CT image; (b) source MR-T2 image; (c) FW-Net; (d) GF; (e) CNN; (f) NSCT; (g) NSCT-PRPCNN; (h) proposed method

Download full size

Fig. 5. Fusion results without and with edge reinforcement branches. (a) Without edge reinforcement branch; (b) with edge reinforcement branch

Download full size

Fig. 6. Fusion results obtained by the proposed method in MR-T1 and MR-T2 image fusion task. (a) Source MR-T1 image; (b) source MR-T2 image; (c) proposed method

Download full size

Module

Convolutional layer

Size

Number of input channels

Number of output channels

Activation layer

Feature extraction module

192

ReLU

Reconstruction module

C10

ReLU

Table 1. Structure of MFEnet

Algorithm MFEnet training and testing algorithms

Training

Input：Training set source image $I_{i}$

Output：Reconstructed image $I_{o}$

1）Randomly select m source images from the training set $\{I^{(1)}, \dots, I^{(m)}\}$

2）Input m source images into the feature extraction module to generate source image features $F$

3）Input $F$ into the reconstruction module to generate a reconstructed image $I_{o}$

4）Use Adam optimizer to update the parameters of the feature extraction module and reconstruction module：

$\nabla_{θ} \frac{1}{W H} ({‖I_{o} - I_{i}‖}_{F}^{2} + α {‖\nabla I_{o} - \nabla I_{i}‖}_{F}^{2})$

5）If the number of iterations is equal to epoch，the training ends，otherwise repeat steps 1）-4）

Testing

Input：Testing set source images $I_{1}$ and $I_{2}$

Output：Fused image $I_{f}$

1）Input $I_{1}$ and $I_{2}$ into the feature extraction module to get the source image features $F_{1}$ and $F_{2}$

2）Input $I_{1}$ and $I_{2}$ into the edge enhancement module to get the source image edge maps $E_{1}$ and $E_{2}$

3）Input $F_{1}$ and $F_{2}$ into the fusion module and the reconstruction module to obtain the intermediate fusion image $I_{f}^{m}$

4）Combine $I_{f}^{m}$ ， $E_{1}$ ，and $E_{2}$ to get the final fusion image $I_{f}$

Table 2. Algorithms of MFEnet training and testing

Method

FMI_pixel

SCD

SSIM

Q_Y

Time /s

FW-Net

CNN

NSCT

NSCT-PRPCNN

Proposed method

0.8182

0.8595

0.8637

0.873

0.8448

0.8741

0.7776

0.8910

1.1123

0.969

1.2690

1.2879

0.5108

0.6612

0.5717

0.6160

0.6350

0.7412

0.0072

0.0167

0.0134

0.0155

0.0122

0.0069

0.6600

0.8407

0.7165

0.7311

0.7575

0.8422

0.0199

0.0912

9.601

3.43

9.601

0.0189

Table 3. Average value of quality evaluation index of 20 fused images

Fusion rule

FMI_pixel

SCD

SSIM

Q_Y

Addition^［35］

Average^［36］

Weighted average^［37］

Max value^［34］

0.8728

0.8689

0.8708

0.8741

1.1951

1.1703

1.2456

1.2879

0.7422

0.7270

0.7337

0.7412

0.0071

0.0068

0.0073

0.0069

0.8116

0.8095

0.8119

0.8422

Table 4. Average value of the quality evaluation index of fusion results with four different fusion rules

Wanxin Xiao, Huafeng Li, Yafei Zhang, Minghong Xie, Fan Li. Medical Image Fusion Based on Multi-Scale Feature Learning and Edge Enhancement[J]. Laser & Optoelectronics Progress, 2022, 59(6): 0617029

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information