With regard to the low segmentation accuracy of planar point sets and poor merging effect of segments in the existing multi-factor segmentation algorithms of point clouds, an improved multi-rule region growing algorithm was proposed in this paper. On one hand, the plane fitting residuals of point clouds were calculated, based on which, the seed condition was set and the segmentation of planar point sets was optimized, so as to increase the segmentation accuracy of planar factors. On the other hand, on the basis of the distance condition, the merging strategy was improved in combination with similarity and volume changes to achieve effective merging of segments. In addition, the threshold parameters involved in this algorithm were set adaptively using the median clustering, Baarda data snooping, and k-means clustering. Furthermore, three different types of point clouds were tested, and the results show that the improved algorithm can boost the segmentation accuracy of planar point sets, and enhance the veracity of segments merging. Compared with other algorithms, the proposed algorithm can take into account both accuracy and efficiency and has better segmentation results.