To solve the problems of low accuracy and poor robustness of traditional video anomaly event detection algorithms, a video anomaly event detection algorithm based on two-stream residual network is proposed.The algorithm uses a combination of deep residual networks, temporal segmentation networks and convolutional fusion strategies.Based on the traditional two-stream network, the algorithm extracts motion information and temporal behavior from single-frame images and multi-frame optical flow images respectively.The network’s depth is further deepened to extend the motion information modeling capability.The temporal features are fully extracted by using the segmented network construction to enhance the effect of processing long-time videos.High-dimensional spatio-temporal features are fused in the middle layer of the network by convolutional fusion method to fully explore the spatio-temporal correlations in videos and obtain final detection results.Experimental results of training and validation on publicly available UCF-Crime and XD-Violence datasets show that the proposed video anomaly event detection algorithm based on two-stream residual network has approximately 10% improvement in accuracy over methods that only use uni-modal network(spatial stream network).The accuracy is improved by 3.2% and 6.1% respectively in comparison with traditional two-stream networks.