This study proposes a multiobject tracking algorithm combining YOLO-V4 and improved SiameseRPN to overcome the low accuracy of existing multiobject tracking algorithms. First, the tracking objects are automatically obtained using the YOLO-V4 network. After creating the template, enter it into the SiameseRPN tracking network. Then, the adaptive background strategy is adopted in the template branch to initialize the template, and the Siamese network is constructed by integrating residual connections. Finally, the results of YOLO-V4 and improved SiameseRPN are used to perform data association through the Hungarian algorithm to achieve multiobject tracking. The experimental results show that the proposed algorithm has better tracking performance than other algorithms. Furthermore, the proposed algorithm can achieve stable tracking under object scale, appearance change, and partial occlusion conditions.