Aiming at the problems of subjective randomness, low repeatability, being time-consuming and low accuracy of traditional chalky rice detection, a new method based on visible spectrogram combined with deep learning algorithm is proposed to meet the requirement of rapid and accurate rice quality parameters in modern agricultural production. In the experiment, CCD color camera was used to obtain the visible spectra of chalky rice and normal rice. Random image transformation methods such as rotation, flipping and contrast adjustment were used to enhance the network training data set to prevent the fitting phenomenon of the depth detection model in the learning process. In this paper, seven deep-level convolution neural network models, including convolution layer, pooling layer, full-connection layer and input-output layer is constructed. The visible spectral image of rice is convoluted and pooled by network model. The characteristic parameters of visible spectral image of rice in convolution layer are obtained by iterative learning training method. The non-linear ReLU activation function is used to accelerate the convergence rate of the effective abstract feature extraction of rice; then the pool layer is employed to obtain the distinguishable semantic features that can express normal rice and chalky rice; finally, the data are transported into the full connection layer. The chalky rice can be identified accurately by classification. The method of rice chalkness detection based on convolution neural network eliminates the complicated steps of feature extraction compared with the traditional method. Because the features extracted by convolution network have more robust expression for specific targets, the algorithm has higher accuracy and less complexity, and the generalization effect is better than the traditional method based on visible spectrogram. The recognition accuracy is up to 90%. The recognition accuracy of SIFT+SVM, PHOG+SVM and GIST+SVM are 70.83%, 77.08% and 79.16% respectively. The proposed method provides a theoretical basis and effective technical means for the realization of automatic and accurate detection of rice quality in modern agricultural production. Therefore, this study has certain theoretical value and practical significance for the realization of artificial intelligence detection of rice quality.