Linear characteristic, which is the basic element of image understanding and analysis, gives important information about the geometric content of images. A fast algorithm based on edge connecting is proposed for detecting line segments in image which contains many artificial objects. The Canny detector is used for edge extraction. An edge connecting strategy which takes into account both the principal orientation and gradient direction is used for selecting candidates of line segments. Noise and curves can be removed from the candidates according to the coherence of principal and straightness errors. After that, the least squares method is used for extracting line segments from the rest candidates. The test result implies that the proposed algorithm is better than the classical component algorithms, such as Hough transform, phase grouping and line detection by principal component analysis, in accuracy and speed.