In micro-expression recognition, directly using the original micro-expression sequence achieves sub-satisfactory results, and the existing algorithms often employ a single feature map rather than fusing multiple feature maps. To address these problems, this paper proposes a new micro-expression recognition algorithm that fuses motion feature maps after extracting the features to obtain more accurate recognition results. The proposed algorithm uses the fused deep learning framework between convolutional neural network (CNN) and long-and-short memory (LSTM) network. Different algorithms are evaluated on the CASMEII micro-expression database. Experimental results show that the proposed method performs better compared with other algorithms.