Breast tumor grading network based on adaptive fusion and microscopic imaging

Pan Huang; Peng He; Xing Yang; Jiayang Luo; Hualiang Xiao; Sukun Tian; Peng Feng

doi:10.12086/oee.2023.220158

Journals >Opto-Electronic Engineering >Volume 50 >Issue 1 >Page 220158 > Article

Opto-Electronic Engineering
Vol. 50, Issue 1, 220158 (2023)

Breast tumor grading network based on adaptive fusion and microscopic imaging

Pan Huang¹, Peng He¹, Xing Yang², Jiayang Luo¹, Hualiang Xiao^3、*, Sukun Tian^4、**, and Peng Feng^1、***

Author Affiliations

¹Key Laboratory of Optoelectronic Technology & Systems (Ministry of Education), College of Optoelectronic Engineering, Chongqing University, Chongqing 400044, China

²College of Computer and Network Security, Chengdu University of Technology, Chengdu, Sichuan 610000, China

³Daping Hospital, Department of Pathology, Army Military Medical University, Chongqing 400037, China

⁴School of Mechanical Engineering, Shandong University, Jinan, Shandong 250000, China

show less

DOI: 10.12086/oee.2023.220158 Cite this Article

Pan Huang, Peng He, Xing Yang, Jiayang Luo, Hualiang Xiao, Sukun Tian, Peng Feng. Breast tumor grading network based on adaptive fusion and microscopic imaging[J]. Opto-Electronic Engineering, 2023, 50(1): 220158 Copy Citation Text

show less

Fig. 1. Block diagram of the algorithm in this paper. (a) ER IHC staining based microscopic imaging procedure for breast cancer pathology; (b) AMFNet

Download full size | View in the Article

Fig. 2. Detail diagram of the AFF method implementation

Download full size | View in the Article

Fig. 3. AFRM method implementation schematic

Download full size | View in the Article

Fig. 4. Visually interpretative results comparisons with SOTA and our method on the breast cancer IHC microscopic imaging

Download full size | View in the Article

Fig. 5. Comparison of visualization results of histopathological images of brain cancer

Download full size | View in the Article

Datasets	Parameter
Datasets	Grade I	Grade II	Grade III	Image size	Total
Training set	268	355	360	224×224	983
Validation set	90	118	120	224×224	328
Testing set	90	119	120	224×224	329
Total	448	592	600	224×224	1640

Table 1. Distribution of the number of ER IHC datasets for breast cancer

View in the Article

Reality	Forecast result
Reality	Positive	Negative
Positive	True positive (TP)	False negative (FN)
Negative	False positive (FP)	True negative (TN)

Table 2. Classification confusion matrix

View in the Article

Model	AMF method			Average acc/%	P	R	F1	AUC
Model	MOO	AFRM	AFF	Average acc/%	P	R	F1	AUC
AMFNet (ViT-AMCNN blocks)	✗	✗	✓	69.00	0.6934	0.6900	0.6903	0.7643
	✓	✗	✗	92.40	0.9240	0.9240	0.9240	0.9423
	✓	✓	✗	93.92	0.9403	0.9392	0.9394	0.9532
	✓	✓	✓	95.14	0.9520	0.9514	0.9513	0.9617

Table 3. Ablation of AMF method in ER IHC pathological microimaging of breast cancer

View in the Article

Model	Grade I acc/%	Grade II acc/%	Grade III acc/%	Average acc/%	P	R	F1	AUC
Inception V3^[36]	74.71	78.19	78.01	77.20	0.7721	0.7720	0.7717	0.8290
Xception V3^[37]	71.82	68.40	76.42	72.34	0.7226	0.7234	0.7226	0.7925
ResNet50^[38]	72.96	73.77	76.08	74.47	0.7524	0.7447	0.7439	0.8085
DenseNet121^[39]	82.80	77.53	81.63	80.55	0.8059	0.8055	0.8047	0.8541
DenseNet121+Nonlocal^[40]	85.88	84.39	81.97	83.89	0.8396	0.8389	0.8391	0.8791
DenseNet121+SENet^[41]	79.55	83.27	84.39	82.67	0.8272	0.8267	0.8266	0.8700
DenseNet121+CBAM^[42]	82.76	81.20	84.80	82.98	0.8307	0.8298	0.8294	0.8723
DenseNet121+HIENet^[43]	86.21	85.59	85.48	85.71	0.8585	0.8571	0.8572	0.8928
FABNet^[15]	86.05	91.29	84.90	87.54	0.8765	0.8754	0.8752	0.9036
ViT-S/16^[27]	54.02	55.87	69.20	60.18	60.37	60.18	0.6023	0.6970
ViT-B/16^[27]	85.87	86.32	84.17	85.41	0.8546	0.8541	0.8541	0.8913
ViT-B/32^[27]	68.60	70.00	73.98	71.12	0.7114	0.7112	0.7107	0.7799
ViT-L/16^[27]	78.98	79.17	81.23	79.94	0.8113	0.7994	0.7987	0.8430
ViT-L/32^[27]	50.35	61.21	59.83	58.36	0.6018	0.5836	0.5774	0.6770
AMFNet (ours)	92.66	97.02	95.12	95.14 $↑$ 7.6	0.9520	0.9514	0.9513	0.9617

Table 4. Tumor grading accuracy of breast cancer ER IHC histopathology microscopic imaging

View in the Article

Datasets	Grade I	Grade II	Grade III	Grade IV	Total
Training set	276	492	873	891	2532
Validation set	91	164	290	296	841
Testing set	91	164	290	296	841
Total	458	820	1453	1483	4214

Table 5. Distribution of the number of brain cancer histopathology image datasets

View in the Article

Model	Metrics
Model	Grade I acc/%	Grade II acc/%	Grade III acc/%	Grade IV acc/%	Average acc/%	P	R	F1	AUC
Inception V3^[36]	84.16	68.11	80.62	78.05	77.76	0.7779	0.7776	0.7766	0.8575
Xception V3^[37]	84.95	72.34	78.31	80.80	78.83	0.7976	0.7883	0.7874	0.8588
ResNet50^[38]	82.00	65.62	66.43	71.12	69.80	0.6963	0.6980	0.6961	0.8091
DenseNet121^[39]	87.05	74.75	74.86	81.75	78.95	0.7978	0.7895	0.7858	0.8625
DenseNet121+Nonlocal^[40]	91.84	81.90	86.70	89.39	87.40	0.8763	0.8740	0.8727	0.9195
DenseNet121+SENet^[41]	96.22	84.97	88.05	90.25	89.18	0.8925	0.8918	0.8911	0.9279
DenseNet121+CBAM^[42]	92.71	82.00	89.04	86.77	87.40	0.8750	0.8740	0.8726	0.9162
DenseNet121+HIENet^[43]	95.70	85.99	87.81	87.50	88.23	0.8851	0.8823	0.8820	0.9244
FABNet^[15]	93.68	87.70	90.97	90.82	90.61	0.9072	0.9061	0.9057	0.9391
ViT-S/16^[27]	65.98	46.98	65.97	69.73	64.21	0.6375	0.6421	0.6359	0.7489
ViT-B/16^[27]	83.90	80.00	87.83	86.83	85.61	0.8618	0.8561	0.8553	0.9028
ViT-B/32^[27]	81.48	62.59	78.93	77.20	75.74	0.7550	0.7574	0.7541	0.8319
ViT-L/16^[27]	70.79	54.49	72.05	73.56	69.32	0.6897	0.6932	0.6902	0.7808
ViT-L/32^[27]	75.49	58.02	73.76	75.08	71.70	0.7152	0.7170	0.7134	0.8084
AMFNet (ours)	98.32	93.38	94.30	93.90	94.41 $↑$ 3.8	0.9451	0.9441	0.9442	0.9611

Table 6. Comparison table of tumor grading accuracy of histopathological images of brain cancer

Download Citation

Save the article for my favorites

Paper Information