An algorithm for road extraction from a remote sensing image is presented. It contains three steps: the road elements and the road segments are firstly extracted by using multiple local statistic characteristics of the image, and the road boundaries are extracted by using energy function. Structural information is described by using statistical analysis in detecting the road elements instead of using lines extracted, to enhance the algorithm’s robustness; the parameters of energy function are determined from the result of above calculation in detecting the road boundaries to make the algorithm’s self-adaptive ability.