Aiming at the problem that pedestrian head is susceptible to illumination changes and occlusion, which leads to low target detection accuracy, a pedestrian head detection algorithm based on two-channel single shot multibox detector (SSD) with multi-scale fusion is proposed. First, a deepth channel is added to the SSD network, and the head features with depth information are fused with the features of the SSD network to form a two-channel SSD network. Then, on the basis of the two-channel SSD network, the high-level feature map with rich semantic information is fused with the low-level feature map to achieve more accurate head location. Finally, the prior frame of SSD is re-adjusted to reduce the computational complexity of the SSD network. Experimental results show that in the case of illumination and occlusion, the detection accuracy of the improved algorithm is improved by 12.9 percentage points compared with the traditional SSD target detection algorithm, and it can effectively solve the influence of illumination changes and occlusion on pedestrian head detection.