In order to improve the accuracy of object classification of point cloud data from airborne LiDAR, an object classification method for multi-source fusion point cloud data based on Point-Net is proposed. Point clouds can effectively represent three-dimensional features of objects, and remote-sensing images contain detailed spectral information. Therefore, a registration and fusion method for point cloud data and remote sensing images is designed to comprehensively utilize their advantages. Meanwhile, considering the lack of neighborhood information in Point-Net, a multi-scale Point-Net classification model for fusion point clouds is also proposed to realize effective classification of fusion point cloud data. The proposed algorithm is verified with point cloud data from urban regions and the classification effect is evaluated by analyzing the classification accuracy and time. Results show that, compared with other methods, the proposed method can effectively improve the classification accuracy of point cloud data, and achieve effective classification of point cloud data in urban areas.