To address the low efficiency in metal surface defect detection, and the problems related to numerous model parameters and low precision, a lightweight detection method based on an improved YOLOv8n was proposed. The partially inverted bottleneck cross-stage partial fusion (PIC2f) module was introduced, replacing the bottleneck module with a partial IRMB bottleneck (PIBN) module. This combination of partial convolution and inverted residual blocks reduced the algorithm’s parameters and enhanced the model’s feature extraction ability. The attention-based intra-scale feature interaction (AIFI) module was applied, integrating location embedding and multi-head attention to improve the model’s small-target detection performance. Lastly, the average pooling down sampling (ADown) module replaced traditional convolution as the feature reduction module, reducing parameters and computational complexity while maintaining detection accuracy. The experimental results show that, compared to YOLOv8n, the PIC2f-YOLO method improves mAP₅₀ by 2.7% on the NEU-DET steel defect dataset and reduces parameters by 0.403 M. Generalization experiments on aluminum sheet surface industrial defects, PASCAL VOC2012 and surface defects of strip alloy functional material datasets also confirm the method’s effectiveness.