A general hierarchical particle filtering framework for designing an optimal proposal distribution is proposed. The essential idea is to augment a second filter′s estimate into the proposal distribution design. Based on this framework,a novel algorithm for robust and efficient visual tracking is given. The algorithm using hierarchical particle filtering procedures at each level and can efficiently extract high-confidence regions through video frames by exploiting the temporal consistency of region confidences. Comparisons with the mean shift tracker and a generic particle filter shows the advantages and limitations of the new approach.