Single part information of building represented by three-dimensional point cloud or model representation is a key information factor in numbers of applications, such as urban planning, municipal management and digital city construction. Using dense matching point cloud generated by aerial images, we propose a new algorithm for rapidly single part of building extraction in complex construction area. On the basic of ground filtering and clustering after horizontal point cloud extraction, the algorithm projects all the point cloud clusters into the two dimensional grid. Non-roof segments are removed based on building fa ade and clusters' geometrical characteristic. Then, topological relationships between clusters computed based on grid images are adopted to generate the range of single part of the building. And the single part point clouds are extracted finally. Experimental results show that the average recall and the average precision of single part of building extraction are 92.6% and 89.9%, and it means that it is efficient for our algorithm to extract single part of building in complex urban area.