Herein, we propose a face template protection method based on multifeature fusion to solve the problems of the poor effect of a single feature template protection algorithm and insufficient template protection before storage. In this method, two feature extraction algorithms were employed to extract different face features to achieve multifeature fusion protection. Furthermore, both features were used as masks in the feature transformation stage. The original image was encrypted using the double random phase mask technology. Before storage, we designed an improved scrambled algorithm based on partitioned magic square transformation to obtain the key from the image using the idea of key generation and properties of the adjoint matrix. To verify the performance of the algorithm, two face databases, ORL and EYaleB, were used. The results show that the scrambling degree of the scrambled algorithm can reach 0.0194 and 0.0187 in the ORL and EYaleB databases, respectively, and the recognition rate of the proposed template protection method reaches 97.12% and 96.90%, respectively. The three characteristics of template protection, namely, irreversibility, revocability, and unlinkability, perform well.