Because the background information of a remote sensing image is complex, traditional morphology makes it easy to change the position and shape of the road when using fixed structural elements to process the image, which affects the accuracy of image segmentation. Therefore, an adapted morphology-based method of road extraction was proposed. First, the nonlinear structural tensor was used to construct adaptive elliptic structure elements and corresponding adaptive morphological operations were created. A morphological top-to-bottom hat transformation was constructed based on road features to enhance road targets. Further, the road was extracted using the maximum interclass variance method. The shape parameters were then set to identify the targets in the image that were either in a road area or not. Finally, the adaptive morphological filtering method was used to remove the non-road targets that were still attached to the road, and the independent road network was extracted. The experimental results show that this method can completely extract the road from the remote sensing images with complex background information and higher extraction accuracy.