Video Nystagmus Classification Algorithm Based on Attention Mechanism

Haojun Zhou; Xiaoli Zhao; Yongbin Gao; Haibo Li; Ruoran Cheng

doi:10.3788/LOP202259.1617001

Journals >Laser & Optoelectronics Progress >Volume 59 >Issue 16 >Page 1617001 > Article

Laser & Optoelectronics Progress
Vol. 59, Issue 16, 1617001 (2022)

Video Nystagmus Classification Algorithm Based on Attention Mechanism

Haojun Zhou, Xiaoli Zhao^*, Yongbin Gao, Haibo Li, and Ruoran Cheng

Author Affiliations

School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201600, China

show less

DOI: 10.3788/LOP202259.1617001 Cite this Article Set citation alerts

Haojun Zhou, Xiaoli Zhao, Yongbin Gao, Haibo Li, Ruoran Cheng. Video Nystagmus Classification Algorithm Based on Attention Mechanism[J]. Laser & Optoelectronics Progress, 2022, 59(16): 1617001 Copy Citation Text

show less

Fig. 1. Convolution process of Mobilenet V2 under different strides

Download full size

Fig. 2. Non-local Block^[24]

Download full size

Fig. 3. SE Block^［25］

Download full size

Fig. 4. 3D Inverted Residual Block

Download full size

Fig. 5. 3D SE Inverted Residual Block

Download full size

Fig. 6. Proposed BPPV nystagmus video classification algorithm

Download full size

Fig. 7. Schematic diagram of video cropping

Download full size

Fig. 8. Relationship between loss value and accuracy of different loss functions and number of iterations. (a) Loss value; (b) accuracy

Download full size

Layer/Stride	Repeat	Output size
Input		3×16×224×224
Conv（3×3×3）/2	1	32×16×112×112
Inverted Residual Block/2	1	16×16×56×56
NL Block/1	2	16×16×56×56
Inverted Residual Block/2	2	24×8×28×28
Inverted Residual Block/2	3	32×8×14×14
Inverted Residual Block/2	4	64×2×7×7
Inverted Residual Block/1	3	96×2×7×7
Inverted Residual Block/2	2	160×1×4×4
SE Inverted Residual Block/1	2	160×1×4×4
Inverted Residual Block/2	1	320×1×4×4
Conv（3×3×3）/1	1	1280×1×4×4
AvgPool/1	1	1280×1×1×1
Linear	1	N Classes

Table 1. Proposed BPPV nystagmus video classification algorithm framework

Mode	0	1	2
Horizontal	Left	Right	None
Vertical	Up	Down	None
Axial	Clockwise	Counterclockwise	None
Intensity	From weak to strong	From strong to weak	None

Table 2. Label description of data set

Algorithm	Number of parameters /MB	Accuracy
C3D^［31］	34.80	0.8443
3D ResNet18^［32］	33.24	0.8518
3D ResNet34^［32］	63.55	0.8717
3D SqueezeNet^［33］	1.87	0.8625
3D ShuffleNetV2^［34］	1.37	0.8502
3D MobileNetV2	2.44	0.8791
Proposed algorithm	2.65	0.9085

Table 3. Performance of mainstream 3D convolutional neural networks on nystagmus video classification dataset

Condition	Accuracy
3D MobileNet V2	0.8791
3D MobileNet V2 +NL Block	0.8922
3D MobileNet V2 +3D SE Inverted Residual Block	0.8853
3D MobileNet V2 +NL Block +3D SE Inverted Residual Block	0.9085

Table 4. Influence of different modules on the model

Label	Precision	Recall	F1-score	N	Label	Precision	Recall	F1-score	N
0000	0.500	1.000	0.667	27	1111	1.000	0.933	0.966	71
0001	0.810	0.708	0.756	123	1112	1.000	1.000	1.000	251
0002	0.854	0.875	0.864	150	1120	0.864	1.000	0.927	74
0010	1.000	1.000	1.000	50	1121	1.000	0.927	0.962	190
0011	0.953	0.968	0.961	371	1122	0.972	0.977	0.975	782
0012	0.955	0.955	0.955	350	1200	0.700	1.000	0.824	45
0020	0.833	0.833	0.833	30	1201	0.867	0.929	0.897	128
0021	1.000	1.000	1.000	100	1202	0.984	0.918	0.950	673
0022	0.976	0.953	0.965	226	1210	0.500	0.400	0.444	13
0101				2	1212	0.857	0.750	0.800	21
0110	0.909	1.000	0.952	84	1220	0.800	0.821	0.810	1280
0111	0.968	0.989	0.978	426	1221	0.830	0.855	0.843	1742
0112	0.947	0.957	0.952	455	1222	0.907	0.874	0.890	2387
0120	1.000	0.933	0.966	60	2000	1.000	0.667	0.800	6
0121	0.952	1.000	0.976	145	2001	1.000	1.000	1.000	29
0122	0.981	0.963	0.972	877	2002	1.000	1.000	1.000	7
0210	0.857	0.857	0.857	51	2010	0.250	1.000	0.400	10
0211	1.000	0.933	0.966	237	2011	1.000	0.818	0.900	32
0212	0.955	0.980	0.967	767	2012	0.500	0.667	0.571	21
0220	0.853	0.871	0.862	1240	2021	1.000	1.000	1.000	29
0221	0.901	0.856	0.878	1746	2022	0.977	1.000	0.988	169
0222	0.878	0.904	0.891	2434	2101	1.000	1.000	1.000	11
1001	1.000	0.979	0.989	249	2102				3
1002	0.953	0.943	0.948	398	2110	1.000	1.000	1.000	12
1010	1.000	0.636	0.778	51	2111	1.000	1.000	1.000	45
1011	0.921	0.946	0.933	114	2112	1.000	1.000	1.000	190
1012	0.889	0.930	0.909	259	2120	1.000	1.000	1.000	6
1020	1.000	1.000	1.000	13	2122	1.000	1.000	1.000	276
1021	1.000	0.862	0.926	136	2201				5
1022	0.986	0.986	0.986	395	2202	1.000	0.857	0.923	32
1100	0.881	0.952	0.915	263	2211	1.000	1.000	1.000	8
1101	0.904	0.881	0.893	540	2212	0.500	1.000	0.667	5
1102	0.921	0.925	0.923	1071	2222	0.976	1.000	0.988	200

Table 5. The performance of the proposed algorithm in each category

Haojun Zhou, Xiaoli Zhao, Yongbin Gao, Haibo Li, Ruoran Cheng. Video Nystagmus Classification Algorithm Based on Attention Mechanism[J]. Laser & Optoelectronics Progress, 2022, 59(16): 1617001

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information