Automatic modulation classifications would play an essential part in wireless spectrum anomaly detection and radio environment awareness. With the breakthrough in deep learning algorithms, this issue can be solved with unprecedented precision and effectiveness by using neural networks. Therefore, a novel neural network termed as Long short-term Convolutional Deep Neural Network(LCDNN) is proposed, which creatively combines the complimentary merits of Long Short-Term Memory(LSTM), Convolutional Neural Network(CNN) and deep network architectures. This model directly learns from raw time domain amplitude and phase samples in training dataset without requiring human engineered features. Simulation results show that the proposed model yields a classification accuracy of 93.5% at high SNRs. Further, the noise sensitivity of the proposed LCDNN model is examined and it is showed that LCDNN can outperform existing baseline models across a range of SNRs. Finally, in order to reduce the computational complexity of the LCDNN model, a ‘compact’ LCDNN model is proposed, which achieves the state-of-the-art classification performance with only 0.6% parameters of the original LCDNN model.