Aiming at the problem that the occlusion interference of unable tracking object effectively in incremental visual tracking (IVT) algorithm, an improved IVT target tracking algorithm is proposed. The problem of a single target appearance model in the IVT algorithm is solved and a hybrid representation method is adopted to represent the target appearance. If the target is not blocked, using the incremental principal component analysis and Gaussian observing noise to represent, otherwise using the continuous uniform probability distribution to represent. The energy minimization method is implemented for mixed model target tracking. Experimental results show that the proposed algorithm has the better ability of anti-occlusion interference during object tracking and it can realize real-time tracking of targets at the same time.