In order to solve the problem of inter-class difference caused by the angle of view and scale change, we propose a method based on histogram of spatio-temporal oriented principal components of three-dimensional (3D) point clouds for action recognition. Firstly, the depth sequences are converted into 3D point clouds sequences. Then, we use a novel image preprocessing method to get new depth sequences. Namely, the sampled depth sequences are limited in spatio-temporal dimension to remove areas with less information, and reduce the redundancy of the input data and the influence of space scale change In order to solve the problem of weak correlation between frames, we adopt histogram of spatio-temporal oriented principal components (HSTOPC) method to describe 3D point clouds sequences and obtain the direction of each point of the 3D point clouds in sequences. For all direction of 3D point clouds in sequences, we use multilayer overlapping segmentation method to obtain HSTOPC descriptor. Finally, we use the support vector machine classifier for training and test. Experimental results on three human action recognition datasets show that the proposed HSTOPC feature descriptor has better robust for noise, rate variations, view change and temporal misalignment, and is able to improve the accuracy of human behavior recognition significantly.