Because of the high missing rate and low accuracy of Tiny YOLOv3 algorithm in the detection of abnormal escalator behavior, an improved Tiny YOLOv3 network structure is proposed for the detection of abnormal escalator behavior. K-means++ algorithm is used to cluster the target boundaries in the data set. The a priori parameters of the network are optimized according to the clustering results to make the training network have a certain pertinence in abnormal behavior detection. The network structure of feature extraction is deepened by using multi-layer deep separable convolution to extract deep semantic information. A scale is added to fuse low-level semantic information to improve the structure of the prediction layer of the original algorithm. Finally, the GPU is used for multi-scale training. The optimal weight model is obtained to detect the abnormal behavior of escalators. The experimental results show that compared with Tiny YOLOv3, the optimized model improves the missed detection rate by 22.8%, the detection accuracy by 3.4%, and the detection speed by 1.7 times. It gives better consideration to the accuracy and real-time performance of the detection.