To address the problems of large noise, multi-view, and difficult to locate and identify digital characters in road scenes, a robust method for recognizing digital characters in road scenes is proposed. According to this method, the maximally stable extremum region algorithm based on the color space and enhanced edge is used first to obtain candidate regions. Then, a geometrically constrained filter is designed and combined with the stroke width transform to filter non-character regions. The convergence function and pooling window of Lenet-5 are improved, and the localized character regions are normalized and input into the network to obtain the final recognition results. According to the experimental results, the recall rate of the proposed method is 90.0%, the comprehensive performance value is 0.89, and the character recognition rate is 88.6%. These results are higher than those of the existing algorithms.