In order to solve the problem of high feature dimension in the feature construction of airborne light detection and ranging (LiDAR) and hyperspectral images for the classification of ground objects, we propose a feature selection algorithm based on extreme gradient boosting (XGBoost) combined with Pearson correlation coefficients (PCCS), named XGB-PCCS. Meanwhile, another feature selection algorithm based on XGBoost combined with sequential backward selection (SBS), named XGB-SBS, is designed to compare with XGB-PCCS. The real data is used to verify the two algorithms designed above. The results show that both algorithms can effectively reduce the dimension of feature sets on the basis that the accuracy of classification results is ensured. As for the XGB-SBS algorithm, the retained feature dimension is 33, the overall classification accuracy is 95.63%, and the Kappa coefficient is 0.943. In contrast, as for the XGB-PCCS algorithm, the retained feature dimension is 25, the overall classification accuracy is 95.55%, and the Kappa coefficient is 0.942. The XGB-PCCS algorithm has low degree of human intervention and short running time, and the retained feature set is compact. In addition, the feature subsets obtained by the two algorithms are compared, and 24 kinds of features with high importance in the multi-modal feature construction of LiDAR point cloud and hyperspectral images are summarized.