Spectral reflectance is considered as the “fingerprint information” of substances, which can reflect the essential properties of the color of substances. The true color of the substance under different light conditions can be accurately restored by obtaining the spectral reflectance information. It has important applications in printing, mural pigment recognition, textile and other scenes. Spectral reflectance reconstruction technology based on multispectral imaging has been widely used in recent years. It has the advantages of non-contact, high efficiency, diversified use scene and so on. The working process can be seen as using the low-dimensional multi-channel response signals output by various imaging devices to reconstruct the high-dimensional spectral reflectivity information of the object. Regression model method is widely used in spectral reconstruction because it has advantages in the model applicability of spectral reconstruction with small samples. The reconstruction accuracy in specific scenes has been continuously improved through the existing regression model reconstruction methods, but the optimization of model parameters in different reconstruction scenes has not been solved. The model is not adaptive enough to achieve the optimal effect of spectral reconstruction in multi-scene. To solve the problem of poor generalization performance of traditional regression model in spectral reconstruction of many scenes, multi-output least-square support vector regression spectral reflectance reconstruction method based on adaptive optimization in multi-scene is proposed to meet the application requirements of optimal spectral reconstruction model in multi-scene. Firstly, multi-output least square support vector regression is used as the reconstruction model, which simplifies the convex quadratic programming problem of traditional multi-output support vector regression. It improves the convergence speed of the model. Secondly, by combining the mean absolute percentage error and Pearson correlation coefficient, a comprehensive evaluation index of the model with adaptive weight is proposed, which can take into account the fitting accuracy and trend of the spectral reflectance reconstruction model. It is used as the fitness function of the sparrow search algorithm to optimize the parameters of the spectral reconstruction model, which can improve the generalization performance of the model. Simultaneously, Chebyshev chaotic map is introduced to initialize the sparrow search algorithm to prevent it from falling into local optimization in the process of optimization. Finally, the spectral reflectance of the test samples is reconstructed by using the reconstruction model with optimal parameters. To verify the effectiveness of this method, 213 standard RAL color cards are used as experimental data. Monochromatic CCD cameras and 10 narrowband filters are used as multispectral imaging systems. Compared with other traditional reconstruction methods, the average spectral root mean square error is reduced by 0.084 0, the average fitness coefficient is increased by 0.69%, and the average chromatic aberration is reduced by 1.23%. To verify the reconstruction effect of this method in different scenes, five different color regions on the temple murals and ancient painted cultural relics in a temple are selected for spectral reconstruction experiments. Compared with others, the average spectral root mean square error of this method is reduced by 0.029 2, the fitness coefficient is increased by 1.29%, and the color difference is reduced by 3.38%. The model parameters can be adaptively optimized for different reconstruction scenes, and better spectral reflectance reconstruction results are obtained in different reconstruction scenes. The experimental results show that this method can meet the requirements of high-precision color restoration of murals and painted cultural relics in practical application.