Aiming at the problem that traffic signs only occupy a very small area in the image and are difficult to accurately identify, an anchorless frame traffic sign recognition algorithm based on the attention model is proposed. The densely connected network DenseNet-121 is used as the backbone network and features are extracted. In order to solve the problem of low accuracy of small traffic signs, an attention model is added to the backbone network to make adaptive adjustments to the space and channel of the feature map. The recognition performance of small traffic signs can be improved by strengthening or suppressing the weight of elements in the feature map. In order to reduce the semantic gap between the encoding path and the decoding path, the residual network connection method is introduced and a semantic connection path is proposed. In order to solve the problem of the imbalance of positive and negative samples in the anchor frame, the detection method without anchor frame can locate the center point of the traffic sign to regression the position and size information of the boundary box. The proposed algorithm is verified on the TT100K dataset, and the experimental results prove the superiority of the proposed algorithm.