In facial expression recognition, the traditional machine learning methods based on the manual feature extraction are time-consuming and less robust. The current convolution neural networks relying on single channel convolution kernel are not sufficient to extract feature, which makes the recognition rates low. We propose a multi-resolution feature fusion convolution neural network, which is combined with two uncorrelated and channels with different depths to extract multi-resolution features. After fusing the two channels feature, a softmax classification is used to classify the facial expression. The experiments on JAFFE and CK+ facial expression databases show that compared with traditional machine learning methods and existing convolution neural networks, the proposed convolution neural network structure model has the advantages of good robustness, strong generalization ability, and fast convergence speed.