The color-based histogram is robust against noise and partial occlusion, but suffers from the presence of the confusing colors in the background. An efficient image sequence tracking method was presented based on multiple cues in Sequential Monte Carlo (SMC). The background-weighted color histogram was combined with edge histogram into SMC for tracking. Color histograms and edge histograms were used to model the object observations likelihoods function. The observations were used to obtain a posterior probability distribution for the location of the object in the sequence images based on SMC. It can be seen from the experiment that the combination of color histogram and edge histogram based on SMC can achieve more robustness and efficient tracking.