This paper improves a global and local feature-based method to overcome problems of the histogram of oriented gradients (HOG), such as the features only characterizing the global gradient feature of motion, lacking local detail information, and having poor performance on occlusion, in the human behavior recognition. The proposed algorithm first uses the background difference method to obtain the human motion region; then, a steerable filter can effectively describe the motion edge features to improve HOG features,therefore enhancing edge details. At the same time, k-means clustering is conducted on speeded up robust features (SURF) to obtain the bag-of-words model. Finally, the merged behavior features are input into a support vector machine (SVM) for classification and recognition. Simulation experiments perform on the KTH, UCF Sports, and SBU Kinect Interaction datasets, showing improved algorithm recognition accuracies of 96.7%, 94.2%, and 90.8%, respectively.