In this study, a manifold regular correlation filter tracking algorithm based on dual-core model context is proposed to balance the tracking speed and accuracy of the algorithm. The main module combines the context-related framework and the relevant filtering algorithm is responsible for the main tracking task, which can compensate for the background information filtered using the cosine window in the relevant filter-learning model. The manifold regular processing of context-related samples can achieve the purpose of penalizing context-related framework and optimizing the main module model. The auxiliary module combines kernel correlation filtering algorithms and convolution features. When the tracking target is occluded, deformed or exceeds the line of sight, the auxiliary module is activated when the tracking confidence is lower than the empirical threshold to prevent the drifting of main module model. The main module has fast-tracking speed and low accuracy, whereas the auxiliary module has slow tracking speed and high accuracy. These modules can complement each other in terms of speed and accuracy. The test results on the OTB2015 and VOT2016 data sets show that the algorithm has better accuracy and robustness than other correlation filtering algorithms.