At present, several mainstream algorithms using color name (CN) all adopt principal component analysis (PCA) to process the feature. However, PCA assumes that the noise of input data must obey Gaussian distribution, which is a conspicuous defect. Aim to address this problem, in this paper, we take robust principal component analysis (Robust PCA) to process CN features. The method projects the original RGB color space to a robust color space–CN space, which means that the input image is stratified to 11 layers according to color name. Then, it processes the CN features by the Robust PCA, so that the mapped image information is concentrated on a few layers, retaining a great quantity of image information and filting out noise. The processed feature is used for Color-tracking frame at the standard benchmark OTB100, and we set up different layers to compare the performance differences of the algorithm. The experimental results show that the success rate increases by 1.0% and the accu-racy increases by 0.9% at OTB100. The result illustrates that the Robust PCA method can better bring color name feature superiority into full play and improve the performance of the algorithm effectively.