In order to better analyze the changes of vegetation and observe the growth status of forestry crops, point cloud data collected by ground-based LiDAR and hand-held LiDAR were adopted in this study to conduct classification research on vegetation through machine learning. At present, classification of vegetation based on feature combination of point cloud covariance matrix has redundancy in its features, and the classification effect of some features is poor. It is mainly reflected in the classification of the boundary of vegetation. To classify vegetation more accurately, this study investigated the point cloud classification based on covariance matrix feature extraction and Fisher algorithm feature selection, and proposed two features of input parameters of support vector machine (SVM) classifier, namely, area feature and pointing feature. In the data collected by the ground-based LIDAR, the weights of the two features that were calculated by Fisher algorithm were 7.25 and 5.78, respectively. The weight of the area feature ranked second only compared to the feature λ₂ with the highest weight of 8.45 (λ₂ is the eigenvalue of the point cloud covariance matrix). The overall classification accuracy using the original features is 99.15%; the overall classification accuracy was improved by 0.75 percentage points after the addition of the new features. Moreover, the classification effect of the junction of tree trunk, ground, and shrub was remarkable. The results showed that the proposed new feature combination has a higher weight coefficient, which can effectively improve the accuracy of vegetation classification. The classification effect of the data collected by the hand-held LiDAR was satisfactory. The overall classification accuracy reaches 99.74% after using the new feature, which verified the strong robustness of the classification algorithm.