The road detection method based on color image exists problems under the extreme lighting conditions and changing road surface types, and the computing resource in moving platform is limited. So, based on the 9-layer convolutional neural network, a fast road detection algorithm is proposed to mix the color image and the disparity images. A new preprocessing method is applied in the data input layer, which can transform the disparity images to disparity gradient maps so as to enhance the representation of roads and reduce the demand for network depth. Two proposed networks are proposed including a double-path convolutional neural network which is used to analyze the characteristics of the convolutional neural network, and a single-path convolutional neural network which is applied to detect the road rapidly. The performance of the proposed algorithm is experimentally compared and analyzed on the KITTI road detection dataset which is divided into a common database and a difficult database artificially. The result demonstrates that, compared with the convolutional neural network method based on color images, the MaxF1 measures on the common database and difficult database improve by 1.61% and 11.58%, respectively, and the detection speed can be 26 frame/s. The proposed algorithm can overcome the impact of the lighting, shadow and the changing road surface effectively.