Deep learning has been a hot spot in the area of machine learning, of which the convolutional neural network is an important component. Based on the deep convolutional neural network and the edge features of characters extracted by auto-encoder, a convolutional neural network of multi-input layers was proposed, which input layers consisted of multi-input with gradient of various directions. In the experiments of handwritten numbers recognition and pedestrian detection, the multi-input network has higher recognition rate compared with the traditional network structure, especially when the number of training time is fewer. This result also provides a proof that multi-input convolutional neural network performed better with appropriate preprocessing.