Benefitting from the low cost-efficiency and broad detecting wavelength of the single-pixel detector, single-pixel imaging is a promising choice for applications such as multi-wavelength, low light imaging. However, to image a scene, multiple measurements are required in single-pixel imaging, which hinders the improvement of its imaging speed, limiting its further development. For the acceleration of single-pixel imaging, an option is to find the better sampling strategy so that the measurements can be greatly reduced by ignoring the relatively unimportant measurements, degrading as little as possible of its imaging quality. To address this problem, deep Q network based single-pixel imaging which considers the scheme of single-pixel imaging as a decision-making process of deep Q network, is proposed for orthogonal transform based single-pixel imaging. It is proved to be an efficient way to find the optimal sampling strategy for deep Q network based Fourier single-pixel imaging and Hadamard single-pixel imaging, nonetheless, more detailed analysis on it for different transforms is needed. For the development of deep Q network based single-pixel imaging, the performance of deep Q network based single-pixel imaging using different orthogonal transform is analyzed comparatively. Derived from the deep Q network based Fourier single-pixel imaging and Hadamard single-pixel imaging proposed before, deep Q network based discrete cosine transform single-pixel imaging and Krawtchouk moment transform single-pixel imaging are proposed. Using structural similarity and peak signal-to-noise ratio as the quantitative image quality evaluation criteria, and artificial planning method for contrast, the reconstructed results of deep Q network based single-pixel imaging using the four kinds of orthogonal transform are quantitatively analyzed. Comparisons among four orthogonal transform based deep Q network single-pixel imaging are also analyzed. The simulation and experimental show that deep Q network based single-pixel imaging is better than artificial planning in imaging quality because deep Q network finds the optimal sampling strategy in a more efficient manner. Deep Q network brings the most significant imaging quality over the others for discrete cosine transform single-pixel imaging while the deep Q network based Krawtchouk moment transform single-pixel imaging overcomes the local effects in natural images, resulting in great improvement in imaging quality. The concentration of spectra is not a perfect criterion of imaging quality but an approximative one. The results provide guidance for the application and improvement of the deep Q network based single-pixel imaging.