In order to deal with two existing problems of correlation filter based target tracking algorithms, a real-time target tracking algorithm based on kernel correlation filtering is proposed, which combines feature fusion with quasi-residual updating, strategy. Firstly, to solve the problem that correlation filters employ the same coefficient to fuse different kinds of features, a correlation filter based on context-awareness with adaptive feature fusion is proposed according to the average correlation peak energy. Furthermore, the correlation filter based on context-awareness and adaptive feature fusion is combined with a Bayes classifier to construct a robust tracker by ensemble learning. Finally, focusing on the use of high-risk updating strategy in correlation filters, an updating strategy that is similar to deep residual networks and an adaptive learning rate updating model based on step function are proposed to prevent tracking model from drifting. The tracker proposed in this paper is compared with another 9 state-of-the-art trackers on OTB2013 and TC128 benchmarks. The experimental result on OTB2013 benchmark is that the proposed tracker ranks first on precision (0.875) and success rate (0.652), which indicates that the proposed tracker is robust and effective.