YOLOv5 (You only look once,v5) is widely used in real-time target recognition because of its fast detection speed and high accuracy. On X-ray security image detection errors or omissions problems with complex backgrounds, multiple scales, and overlapping. By improving the attention mechanism, a new feature fusion strategy is developed based on the YOLOv5s network structure. This study proposes a YOLOv5s-AFA object detection network with an adaptive feature fusion technique and an attention mechanism. In the shallow layer of the network, an extended receptive field module and an improved spatial attention mechanism are introduced, whereas the improved channel attention mechanism is introduced in the deep layer. The new feature fusion technique can output three feature maps of varying depths at a time and fusing shallow spatial and deep semantic information using adaptive learning weights to improve the network learning. The target results on the X-ray security image dataset show that the false and missed detection rates of the YOLOv5s-AFA network decrease considerably compared with other compared networks.