The traditional correlation filter tracking algorithms have low precision when the target is in fast motion, occluded, or under complex background. To solve the problem, a correlation filter target tracking algorithm with adaptive context sensing was proposed. Based on the framework of the correlation filter algorithm, the boundary effects brought by the cyclic shifts and the fixed learning rate were mainly improved. Firstly, an adaptive sampling strategy based on response map was used to sample the context information in the classifier training stage. Then, a segmented learning rate adjustment strategy was used to make the algorithm more adaptive to the changes of target. Finally, the performance of the proposed algorithm was verified on the standard data set. The experimental results showed that the proposed algorithm can improve the tracking precision of DCF and SAMF algorithms. It not only has better robustness in the case of fast motion, occlusion, and complex background, but also can be used as a framework integrated into most of the correlation filter trackers.