To address the fusion problem of time sequence features in video classification, this paper proposes a new three-dimensional (3D) squeezing excitation (SE) network structure module that is constructed by combining the SE network in a two-dimensional convolutional neural network (CNN) with a 3D convolutional residual network. The new module adds an extra time-dimension coefficient to the coefficient set of a directly transformed 3D SE module, allowing it to record the changes in the motion trajectories of the research objects on time trajectories. The proposed module can not only record the characteristics of a specific time point, but also strengthen the relevance of multiple time points. To assess the effectiveness of the module, an SE network with a spatial and temporal latitude was used to perform character-action-behavior recognition. The experimental results indicate that the module can accelerate the loss convergence and effectively improve the accuracy of video classification.