Referring to the limitation of driver face detection algorithm based on single feature in detection precision and reliability, a novel fusion algorithm of driver face detection is proposed. Firstly, an improved face detection algorithm based on Haar-like feature is used to detect the possibly existing face region in the whole image. Then, the detected region is extended adaptively and a face detection algorithm based on skin color feature in YCbCr space is used to detect the face again in the extended area. Finally, double matching verification is made by the defined area coincidence degree and geometric prior knowledge of human face and fusion detection of driver face region is achieved by establishing relevant location rules. Experiment results in various complicated road conditions show the effectiveness of the proposed algorithm.