In order to solve the problem of vulnerable to attacks caused by symmetry in traditional cryptographic systems, a new single-channel color image encryption using detour cylindrical diffraction and color space conversion algorithm was proposed in this paper. In the process of encryption, firstly, the color image was transformed into a single channel image in YCbCr4:2:0 format, then the image was encrypted by twice cylindrical diffraction and phase truncation operations. In the process of decryption, the keys were combined with the cipher text, and the color original image was reconstructed after twice inverse cylindrical diffraction. Because the cylindrical diffraction process was an asymmetric process, the proposed algorithm can overcome the symmetry characteristic of the encryption system based on plane diffraction. Applying it to the encryption system based on phase truncation can further improve the security of the encryption system. The simulation results show that the algorithm can effectively encrypt the color image with single channel, which not only can restore the original color image with high quality, but also has high security.