In order to overcome the inaccurate tracking problem of Kalman filter in the case of long distance and occlusion, a multi-target real-time tracking framework is designed. In the framework, neural network is used to predict the state of the target object in three-dimensional space, Hungarian algorithm is used for data association frame by frame, and the trajectory management module is designed to manage the corresponding trajectory to realize multi-target tracking. Compared with the traditional framework, this framework does not need to perform Kalman filtering in the image space. It can not only track occluded targets accurately at high frame rate, but also has excellent performance for long-distance targets. The performance of the framework on Kitti dataset is as follows: multi-target tracking accuracy is 79.22, multi-target tracking accuracy is 78.33, most of the lost number is 54.19, most of the tracking number is 13.21, ID transition number is 16, the running speed is 39 frames per second. Compared with the traditional Kalman framework, the accuracy of the designed framework is improved by 11%, and the anti occlusion performance is also greatly improved.