In order to extract human action features effectively under the panoramic view, we propose a novel PMK algorithm with adaptive partitioning based on the original shape context feature matching algorithm .First we combine optical imaging principle with the projection characteristics of human body under the panoramic view, and utilize the second moment to correct the principal axes direction of human contour. Then do the uniform sampling with the edge points, and extract the shape context feature at each sample point. In the process of matching sampling points, through the analysis of the distribution characteristics of sampling points in the high dimensional feature space, we introduce adaptive partitioning to Pyramid Match Kernel (PMK) algorithm to improve the convergence strategy. According to the range of data points at each dimension, the improved algorithm can adjust convergence coefficient the data adaptively, thus achieving the consistent convergence speed of the points set at each dimension. Finally we perform an experiment on fall indoor detection to verify the reliability of the proposed algorithm, and the K-means clustering algorithm is used for classification,the recognition rate can reach 92.9%. The results show that the improved feature extraction algorithm can provide the guarantee for the stability of the intelligent monitoring system.