For the shortcoming of the real-time performance of YOLOv3 algorithm in object detection, we propose an improved network structure and a new method for video object detection adapted to real-time object detection. Firstly, the proposed k-means-threshold (k-thresh) method makes up for the problem of its sensitivities to the initial position of the cluster center, and performs cluster analysis on a data set including three categories to select more appropriate anchor boxes. Then, the 4×down-sampling and 8×down-sampling feature maps are stitched together into the third layer detection layer to improve the detection accuracy of the object and increase the the mean average precision of the YOLOv3 algorithm by 2%. Finally, the camera captures the image and the excellent detection data obtained in the previous period to predict the target of the new image and adds a re-detection threshold to improve the smoothness of video detection. The experimental results show that the proposed improved YOLOv3 network improves the detection accuracy and the real-time performance, the maximum frame rate reaches 64.26 frame/s in 30 min of real-time detection, which is 4 times faster than the original YOLOv3 algorithm.