ing at the problems of interference and noise in image recognition of aerial asphalt pavement, a pavement crack recognition algorithm applied to aerial image is put forward. According to the difference of gray level distribution of the surface area and the roadside landscape area, a method of regional growth based on multi-directional fitting and threshold segmentation in HSV color space for road region segmentation is proposed. The single channel pavement which contains integral crack information is extracted, the large area of interference is eliminated by the improved morphological filtering, and an edge detection algorithm based on saliency analysis to recognise the crack fragment of pavement is proposed, realizing the distinction between complex cracks and pavement texture noise. The images with crack are screened automatically and the crack length is marked and calculated combined with human eye assistance observation. The experimental results show that the proposed method can effectively remove the interference and noise in the image, and well identify asphalt pavement cracks. The precision of crack width is 9.7 mm. The classification accuracy is over 80.0%. The accuracy of length measurement is over 75.0%.