In view of the difficulty of automatic port recognition, the ship-wharf-port progressive recognition model is proposed by combining deep learning and geospatial analysis on high-resolution visible light remote sensing images. Firstly, the constructed wharf sample data set is enhanced, and the enhanced data set is used to train the YOLO v3 algorithm. Then, the multi-scale recognition is carried out by the sliding window on the large remote sensing images, and the underlying features of the images are obtained to calculate the wharf categories and pixel coordinates. Finally, the locations of wharves are transformed into geographical coordinates, and the Getis-Ord Gi ^* statistical method is used to analyze the hot spots. The classical density clustering method is used to identify and extract the locations and ranges of ports. The recognition comparison results in the experimental area show that the proportion of port basin recognition by improved model reaches 82.79% at aggregated threshold of 1000 m .