ing at the problems of object tracking failure caused by occlusion and out of view in long-term tracking, we propose a long-term object tracking algorithm based on feature fusion to improve the speed and robustness of object tracking. First, the features of histogram of oriented gradient, color space and local sensitive histogram are fused to enhance the robustness of the algorithm in complex cases, and the fusion feature dimension reduction is carried out to improve the object tracking speed. Then, an additional one-dimensional scale correlation filter is used to obtain the optimal scale estimation of the object, and the computational complexity is reduced by quadrature rectangle-factorization. Finally, the object detection threshold is adaptively determined. When the object occlusion or out-of-view causes the failure of object tracking, the object region proposals can be extracted by EdgeBoxes, and object position is re-directed by using structured support vector machine to complete the long-term tracking of object. Experiments are conducted on standard tracking datasets OTB2015 and UAV123. Experimental results show that the average accuracy of the proposed algorithm is 5.0% higher than that of other optimal algorithms, the average success rate is increased by 2.6%, and the average object tracking speed is 28.2 frame/s, which meets the real-time requirements for tracking. In the case of object occlusion and out of view, the proposed algorithm can track the object continuously and accurately.