The contact between distortion of visual perception characteristics and quality of stereo image is explored, and a new stereo image quality assessment model is proposed in combination with visual perception characteristics and random forest learning method. Firstly, according to human visual system, a series of visual perception characteristics are extracted as features to measure the degree of distortion stereo image, called perception characteristics sets. Secondly, random forest learning method is utilized to simulate HVS mechanism and fuse these features, and then a model is built for evaluating stereo image quality. By applying the proposed model to stereo image test database, experimental results demonstrate that proposed model is consisted with human perception well.