The existing classification algorithms for benign paroxysmal positional vertigo video nystagmus have the following shortcomings. The features extracted manually are subjective and limited; the feature extraction of axial rotation of eyeballs is difficult; it can only distinguish between normal people and patients or classify simple nystagmus. To overcome the above shortcomings, a video nystagmus classification algorithm based on attention mechanism is proposed. Based on the lightweight model three-dimensional MobileNet V2, a network is used for feature extraction, and the global spatiotemporal attention module is introduced at the lower level of the network with rich global detail features and spatiotemporal information to integrate the spatial information of nystagmus and the temporal information between frames. The attention mechanism of the spatiotemporal channel is introduced to the high-level network to screen high-level semantic features. The cross entropy loss function with category modulation coefficient is used to train the network, which effectively alleviates the problem of imbalance in several categories. Experiments were conducted on 66 types of video nystagmus datasets provided by the Eye and ENT Hospital of Fudan University. The classification accuracy of the proposed algorithm reached 90.08%, and the average accuracy, recall, and F1-score of each category were 90.50%, 92.00%, and 90.40%, respectively, indicating the superiority of the proposed algorithm.