Aiming at the problem of information security in remote sensing images and color images transmission, a method aimed at encrypting color remote sensing images using vector operations and secondary image phase masks is proposed. In this encryption, the optical coherent superposition principle is used to separate and overlay the original image into two phase plates on the three channels R, G and B, respectively. Then the dual random phase encoding in the Fresnel domain is performed using the secondary image phase masks to encrypt one of the phase plates. Finally, the Kronecker product of two random matrices is directly perform randomization on the encoded image, achieving multilevel encryption of color remote sensing images. Experimental results show that the algorithm has a high key sensitivity, and it has good robustness under Gaussian noise attack with intensity coefficient less than 0.04 and statistical analysis attack. The ciphertext can resist the chosen-plaintext attack and has stronger security than the traditional double random phase encoding algorithm. The peak signal-to-noise ratio and correlation coefficient of the decrypted remote sensing images can reach 31.92 dB and 0.988 8. This encryption method provides a new idea for the encryption of a large number of remote sensing images of the same size.