In this study, a training model was constructed to identify several abnormal behaviors in video sequences. A convolutional neural network (CNN) was used to extract features, and the features were then optimized using a gradient-based optimization algorithm known as Adam algorithm. The adaptive pooling layer was introduced for feature discrimination to reduce the computational complexity of the network and rapidly identify abnormal behaviors in video sequences. The recognition rate reaches 87.6% after using the Adam algorithm for model optimization. The recognition rate reaches 91.9% when the adaptive pooling layer is introduced. CNN is faster and more accurate than the improved dense trajectories and the two-stream networks in detecting abnormal behaviors in video sequences. Compared with the temporal segment networks and temporal relation networks, the CNN has a lower accuracy but a faster speed.