In order to improve the accuracy of the pedestrian detection, identification and tracking, a pedestrian tracking algorithm is proposed based on human body characteristics identification and Kalman filter.The algorithm uses the recognition method that is based on the human body feature and behavioral characteristic gesture to automatically identify pedestrians.Behavioral characteristic gesture recognition algorithm uses a front and rear frame pixel probabilistic matching algorithm with Gaussian mixture model, and improves probability of success for human recognition by using human body features.Then, a histogram of oriented gradients is used for characterization of target detection.Finally, the target moving trajectory is predicted by Kalman filter.Experiment was made to the two infrared pedestrian detection and tracking methods, and the result showed that, the algorithm can more accurately capture the human target from moving object for real-time tracking.