To address the problems of blurring and insufficient sample size in sonar images, an improved sonar image target detection algorithm is proposed based on YOLOv5. The algorithm uses geometric filtering, vertical flipping, and other methods to enhance the sonar image dataset. The fusion attention mechanism module is added to make the algorithm better focus on the features of small targets in sonar images. At the same time, in response to the problem that most target detection algorithms currently run on the cloud and cannot achieve real-time sonar image detection, this paper uses lightweight network replacement and NCNN edge porting technology. It adopts the GSConv module in the neck network to successfully transplant the algorithm to the ZYNQ platform, realizing real-time detection of sonar images on the embedded end. After experiments, the algorithm proposed in this paper reduced the parameter quantity by 56%, increasing map50 and map50-95 by 2.2% and 2.5%, respectively. The algorithm’s performance has significantly improved, proving that the method proposed has certain feasibility and effectiveness in lightweight sonar image target detection tasks.