Aimed at the human face contour tracking in video image sequences, a contour tracking algorithm is proposed in which an improved level set method is introduced into the particle filtering framework. The face contour is represented by zero level set function and approached by evolution of the level set function. To deal with the partial occlusion, the shape prior is added to the updating process to constrain the curve evolution. This algorithm can simultaneously track the affine motion of the face region and deformation of the face contour in the framework of particle filtering. Experimental results show that the proposed algorithm can track the face contour accurately and be robust to illumination changes, background interference and partial occlusion.