We propose a new method for stereoscopic image comfort assessment based on convolutional neural network, which does not need to extract specific manual features from images in advance according to specific tasks, but simulates hierarchical abstract processing mechanism of human brain to extract image features autonomously. This method adopts three channel convolutional neural network structure, and the input data sets of the three channel are obtained by reducing the dimension of the original data samples through principal component analysis, and chopping the original data samples into two size image patches (32×32, 256×256), respectively. The network structure of each channel is designed according to the input data sets. In addition, the classification accuracy of this method is improved by introducing dropout and local response normalization, etc. With rectified linear unit as the activation function and Softmax as the classifier in the output layer, experiment results on 400 stereo image samples in TJU database with different comfortable levels show that, the correct classification rate of this method is 94.52%, which is higher than that of the extreme learning machine and support vector machine.