The point cloud scanned by terrestrial laser scanning contains mass data. Not all of these data are useful in the process of application, especially for the building point cloud, the building can be described when the building profile is determined. Therefore, the geometrical characteristics of adjacent points are used to extract features of the building point cloud. Firstly, the proposed algorithm uses the k-nearest neighbor search algorithm to search the adjacent points of one point. The normal vector and datum plane are determined according to the adjacent points. The characteristics of normal vector angle between the probe points and the adjacent points are used to determine the building boundary. Secondly, the total least squares and weighted principal component analysis are used to improve the random sample consensus algorithm. The point clouds on both sides of the fold boundary are determined by the improved algorithm. The characteristics of boundary are used to probe the building fold edge. The results show that the proposed algorithm is fast and less redundancy, and can be used to extract feature lines of the building with more than 90% eliminating rate of the invalid point cloud.