Poor lighting conditions lead to a high accident rate during night driving. In order to reduce the incidence of night traffic accidents, various auxiliary driving technologies such as ultrasonic ranging, millimeter wave radar and visual auxiliary driving are widely used. Infrared thermal imaging technology based on the thermal radiation of object and reflection imaging with certain penetrability is less affected by the weather and light conditions at night. Human targets within the vision field can be accurately captured by infrared thermal imaging technology, which is convenient for pedestrian detection. In addition, the cost of infrared imaging equipment has been decreased in recent years, making it possible to be mounted on vehicles. Therefore, the fusion of infrared thermal imaging technology and pedestrian target detection algorithm based on deep learning is of great research significance and with a broad market application prospective in vehicle auxiliary driving. In this paper, a pedestrian detection model based on night infrared image is proposed for night driving, which can detect pedestrians on the night road in real time. This study can be applied to the field of auxiliary driving for early warning and active braking provided to drivers, reducing the probability of night driving accidents and providing higher security for vehicles and pedestrians.
Aiming at the problems of low accuracy in infrared pedestrian detection for small targets at night, large committed memory of network model, and the difficulty of real-time detection in auxiliary driving due to the low model detection speed, a lightweight pedestrian detection neural network called YOLO-Person is proposed for night infrared images. Firstly, the MobileNetV3 lightweight network is used as the backbone network, while the multi-scale fusion target detection layer is used as the prediction module to solve the problem of large model size and slow inference speed, which greatly reduces the amount of model calculation and obtains a preliminary lightweight network model. Furthermore, by adding the spatial pyramid pooling module and the detection layer with smaller receptive field in the network, the representation ability is enhanced to solve the problem of unbalanced pedestrian target scale in the dataset and improve the infrared pedestrian detection accuracy. Finally, channel pruning is used to reduce the number of channels in the feature map, and the final network model YOLO-Person is obtained. The lightweight model YOLO-Person is verified on the pedestrian dataset of night infrared images based on Jetson Nano mobile development platform.
A lightweight model YOLO-Person is proposed for night infrared pedestrian detection (Fig. 1). Firstly, MobileNetV3 lightweight network is used as the backbone network, and the multi-scale fusion detection layer is used as the prediction module. Although the accuracy is reduced by 1.2%, the speed is increased by 34 frame/s, and the model size is reduced by 151 MB (Table 1), which indicates that the lightweight of the night infrared pedestrian detection model is preliminarily realized. Secondly, aiming at the problem of unbalanced pedestrian target scale in dataset, spatial pyramid pooling module (Fig. 2) and small receptive field detection layer are added in the network, through which the accuracy is improved by 3.3%, the speed is reduced by 23 frame/s, and the model size is increased by 5.1 MB (Table 2). Moreover, the model is pruned (Fig. 3) to reduce a large number of redundant channels (Fig. 6). When the pruning rate is 95%, the number of model channels, accuracy and model size achieve balance and optimization (Table 3). In addition, the model is fine-tuned to obtain the final lightweight model YOLO-Person, which reaches the accuracy of 92.2%, the speed of 69 frame/s, and the model size of 11.7 MB (Table 4). Finally, the model is deployed on the Jetson Nano mobile development platform to verify the detection effect (Fig. 7), and the test results of three networks are compared. The lightweight model YOLO-Person gets the best results: the accuracy of 92.2%, the speed of 12 frame/s, and the model size of 11.7 MB (Table 5).
A lightweight model YOLO-Person for night infrared pedestrian detection is proposed in this paper. Firstly, MobileNetV3 lightweight network is used as the backbone network, and the multi-scale fusion detection layer is used as the prediction module to achieve the preliminary model lightweight. Secondly, spatial pyramid pooling module and small receptive field detection layer are added to improve the detection accuracy of small targets. Finally, the model parameters are greatly reduced through channel pruning, and the final lightweight model YOLO-Person is obtained. The experimental results show that the detection accuracy and speed of YOLO-Person model reach 92.2% and 69 frame/s, respectively, meeting the requirements of real-time pedestrian detection. The YOLO-Person network model is deployed on the Jetson Nano mobile development platform, where the detection speed of 12 frame/s exceeds that of YOLOv3 and approaches that of YOLOv3-tiny, which further verifies the superiority of the proposed method. By optimizing the network structure and increasing the effective functional network layer, the detection accuracy of the model will be further improved in the future research.
