A digital hologram watermarking with a phase encryption feature is proposed based on a binary optical phase retrieval algorithm. The watermark has been encoded with the binary optical phase algorithm and encrypted with a binary phase key. Comparing to the digital hologram watermarking with binary phase structure under the plane illumination, the security of the proposed method is greatly improved by using a binary phase key while the quality of decrypted watermark with large information contents is almost maintained unchanged. The decrypting processing is independent of original host image. The quality of decrypting watermark under the different sizes of encryption keys has been discussed. The simulation results prove the theoretical analysis.