Tracking of target is easy to be disturbed due to sheltering, changing of illumination, or interference of noise under complex background.To solve the problem, a robust target tracking method based on binary channels of infrared and visible frames was proposed.Multiple features of the two channels were extracted, and the sparse sampling feature of compressed sensing was used to eliminate the non-negative hypothesis in sparse tracking algorithms.A new algorithm of compressed sensing was presented based on particle filter framework, and an adaptive method for target template updating was given, which was based on the Bhattacharyya coefficients.Experiments were made using multiple groups of image sequences with complex terrain.The results show that: the proposed algorithm has better robustness, higher precision and lower computation burden than the other tracking algorithms, which can achieve a stable tracking of the image target in complex environment.