One optical encryption scheme with double secret keys based on the Toeplitz matrix and axial distance in the computational ghost imaging (CGI) is proposed. In this scheme, multiple random speckle patterns can be obtained by the transmission of a single random phase mask in legitimate users. At the receiving terminal, the image is decrypted with compressive sensing technique. The numerical simulation and experimental results indicate that the authorized users can recover the original image completely, but the eavesdroppers cannot obtain any information of the original image even if the eavesdropping rate is up to 60%. The use of double secret keys can effectively improve the security of this scheme.