To solve the problems of illumination variation, occlusion, and scale variation during object tracking, we propose a multi-scale context-aware correlation filter tracking algorithm based on channel reliability. First, we extract the histogram of the oriented gradient (HOG), gray features, and color name (CN) features as the appearance model of the object, which can enhance the robustness of the tracking algorithm in a complex scene. Second, the single-channel context-aware correlation tracker is independently trained by applying the related channel feature samples. The channel reliability factor is applied to evaluate the confidence of each channel. Then, the final response map of the multi-channel context-aware correlation tracker comprises the response maps and the channel reliability values of all the channels, and it is used to accurately locate the object. Finally, the scale pool method is applied to estimate the optimal position and scale of the object. When compared with the results obtained using the state-of-art trackers, the experimental results show that the proposed algorithm can effectively tackle the illumination variation, occlusion, scale variation, and other complicated factors, and achieve relatively high tracking accuracy and success rate. The overall performance of the proposed algorithm is superior to those of other algorithms.