Lane detection plays an important role in automatic driving. It is the premise of lane keeping, lane departure warning and high-level automatic driving. Lidar has become a new direction in this field because it can generate more spatial three-dimensional (3D) information and is less affected by bad lighting, shading and other conditions. Currently, lane detection is mainly based on deep learning. Compared with traditional methods, it has higher detection accuracy and better robustness. The key of 3D lane detection with lidar based on deep learning is how to extract and utilize the feature information of lane point cloud completely and efficiently, otherwise it will not be able to cope with various lighting conditions and challenging scenes, which will have a great impact on the realization of automatic driving function. Therefore, how to fully extract and utilize the feature information of point cloud is the key to improve the accuracy of lane detection.
The proposed lidar lane detection network, LLDN-AGDP, consists of three parts, i.e., bird’s eye view (BEV) encoder network, backbone network, and detection head. In the BEV encoder part, the original 3D point cloud is projected into a two-dimensional (2D) pseudo-image by a point projector, and the feature is extracted by ResNet34 with global feature pyramid network (GFPN) (Fig.2). By fusing multi-level features of different scales, a feature map with globally relevant multi-level semantic information is constructed. In the backbone network part, firstly, through the efficiency mobile convolution (E-MBCONV) module (Fig.4), the local information between the window pixels is exchanged to generate better downsampling features and enhance the network representation ability. Then, the feature map is input into the dual-pathway module and the fusion module (Fig.5), and the low-level high-resolution texture features are compressed into high-level abstract semantic features, thus reducing the computational complexity. When learning finer low-level texture details, the compressed high-level abstract semantic features can be used as prior information, thus reducing the difficulty of global feature extraction. Moreover, the adaptive multi-order gating (AMOG) module (Fig.6) is embedded in the dual-pathway module and the fusion module, and the multi-order spatial interaction is carried out by using the rich context information between the cross-level feature maps, so that the network can adaptively extract lane lines. Finally, the lane lines are classified and located by the detection head.
The proposed LLDN-AGDP network is tested and evaluated on the K-Lane test set (Tables 1 and 2). Compared with the comparison networks, the average confidence F1 score and average classification F1 score of LLDN-AGDP are 84.7% and 83.6%, respectively, and the performance of LLDN-AGDP is greatly ahead of the lidar lane detection network using convolutional neural network for the backbone. Meanwhile, LLDN-AGDP outperforms LLDN-GFC, RLLDN-LC and other lidar lane detection algorithms with advanced global feature extraction network in all kinds of roads and scenes. Under bad lighting and severe occlusion conditions, the average confidence F1 score is increased by 2.7 and 3.5 percentage points, respectively, and the speed is at the same level as that of the benchmark network LLDN-GFC. Through the visualization of attention scores, the robustness of each model under occlusion conditions is compared and analyzed (Fig.10). The comparison between LLDN-AGDP and other network attention visualization shows that LLDN-AGDP can pay more attention to the areas with lane characteristics and show stronger interest in the lane lines in the blocked areas. Then, the effectiveness of the proposed innovation module is further analyzed (Table 3). The results show that after adding GFPN structure to the network, it can effectively fuse the features of shallow strong position information with the features of deep strong semantic information with global correlation, which brings stronger representation ability to the network. After the introduction of the dual-pathway structure, the network can make full use of the differences between different levels of features to further dig deep-seated global information. After the E-MBCONV module is added, it is beneficial to alleviate the attention limitation of local windows in the dual-pathway structure and realize the interaction of information in the windows. After adding the AMOG module, the feature capture ability of the network is stronger by using multi-order spatial interaction of context information.
A 3D lane detection algorithm LLDN-AGDP for lidar based on adaptive multi-order gating and dual pathways is proposed. GFPN structure is proposed in BEV encoder to enable the network to effectively fuse texture features and semantic features of different levels, and pay attention to the global information of lane lines at the early stage of the network. In the part of backbone, a dual-pathway global feature extraction network and AMOG module are proposed, which reduce the computational complexity and the difficulty of deep global feature extraction through the interactive and complementary information flow structure of the two pathways. The AMOG module can make use of rich context information and adaptively aggregate the more representative features of lane lines to improve the detection accuracy of lane lines. Moreover, the E-MBCONV module, which can effectively exchange the local information among the pixels in the window, is introduced. The test results on K-Lane test set show that the average F1 score of the proposed algorithm can reach 84.7% under different road conditions and scenes, which is 2.6 percentage points higher than that of state-of-the-art model, and the F1 scores under bad lighting and severe shading conditions are increased by 2.7 and 3.5 percentage points, respectively. Finally, LLDN-AGDP algorithm is deployed on a real vehicle to verify its engineering application value.
