According to the problem of the severe detail loss of the disparity map generated by the current convolutional neural network methods, a structural improvement method is proposed. The 4 layers convolutional structure of the feature extraction part from original network is added to 7 layers to maximize the accuracy. And, the proposed dual pyramid structure is introduced to the network to combine the multi-scale down-sampling information with the feature information, which keeps the details of the original input images. Experimental results show that the error rate of the improved network reduces from 3.029% to 2.795%, and the generated disparity maps have better connectivity.